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ABSTRACT 



This monograph, part of a series on student assessment, 
reports on the results of a national survey that examined institutional 
support for undergraduate student assessment. It provides a national profile 
of student assessment initiatives at baccalaureate institutions and compares 
it to assessment practices and support patterns found in all types of 
postsecondary institutions. The study also offers advice for administrators, 
faculty, and staff in baccalaureate institutions who are designing student 
assessment approaches and support processes that are likely to foster 
institutional improvement. Also included is the survey instrument used to 
gather study data, which institutions can use to examine patterns for student 
assessment on their own campuses. The monograph is organized in nine 
sections: (1) "Introduction and Overview"; (2) "Perspectives on Student 

Assessment in Higher Education"; (3) "The Influence of External Groups"; (4) 
"Approaches to Student Assessment"; (5) "Organizational and Administrative 
Support Patterns"; (6) "Assessment Management Practices and Policies"; (7) 
"Uses and Impacts of Student Assessment"; (8) "Key Relationships"; and (9) 
"Executive Summary: Research Results and Recommendations." Appended is the 
survey instrument, the "Inventory of Institutional Support for Student 
Assessment for Baccalaureate Institutions," and information on the 
construction and content of variables. (Contains 42 references.) (SM) 
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1. INTRODUCTION AND OVERVIEW 



This monograph reports the results of a national research survey that 
examines institutional support for undergraduate student assessment. It 
provides a national profile of student assessment initiatives at baccalaureate 
institutions; useful advice for administrators, faculty, and staff who are 
designing student assessment approaches and support processes that are likely 
to foster institutional improvement; and an instrument that institutions can use 
to examine patterns for student assessment on their own campuses. 

This report is a result of the National Center for Postsecondary Improvement (NCPI) 
research program examining organizational and administrative support for student assessment. In 
1997-98, NCPI researchers conducted a major literature review followed by a national survey on 
institutional support for student assessment. While other dissemination efforts have focused on a 
research audience, this report is specifically written for administrators, faculty, and staff in 
baccalaureate institutions who are involved with student assessment on their campuses. 

Purpose of the Monograph 

Administrators and faculty leaders have little credible and verifiable evidence to guide their 
planning and decision making regarding student assessment. The literature on student assessment 
offers many descriptions of student assessment practices at a variety of postsecondary institutions 
(Banta & Associates, 1993; Banta, Lund, Black, & Oblander, 1996) and a number of limited 
surveys have collected information concerning student assessment measures and methods used by 
focused groups of institutions (Cowart, 1990; El-Khawas, 1988, 1989, 1990, 1991, 1992, 1993, 
1995, 1996, Johnson, Prus, Andersen, & El Khawas, 1991). There has been comparatively less 
consideration of how colleges and universities can develop an effective institutional strategy for 
assessment or of organizational and administrative practices that support internal engagement in 
assessment and enhance the use of student assessment data. Guidelines for planning and 
implementing student assessment efforts are available (American Association for Higher Education 
[AAHE], 1992, Ewell, 1988a, 1988b; Rossman & El-Khawas, 1987) but largely missing is any 
systematic examination of the relationships among various ways institutions have approached and 
supported student assessment and the likelihood of their reaping positive impacts from student 
assessment efforts (Banta et al„ 1996; Ewell, 1988b; Gray & Banta, 1997). Furthermore, there 

has been little research focused specifically on student assessment within baccalaureate institutions 

/ 

Understanding the national profile of undergraduate assessment activities and support 
patterns found in baccalaureate institutions is important for several reasons. First, it is helpful for 
institutions to know what their peers are doing in relation to student assessment. How are other 
institutions responding to external demands for student assessment? What approaches to student 
assessment are they using? How are they supporting and promoting student assessment through 
their practices and policies? How are they using student assessment data? What impacts has 
student assessment had on faculty as well as institutional and student performance? Finally, if 
student assessment activities do make a difference in terms of improved teaching and learning on 
campuses, how does that happen? What external influences, institutional approaches, and 
organizational and administrative support practices encourage internal involvement in student 
assessment and positive impacts from student assessment data? 

This monograph will answer these questions through three primary purposes. First, it will 
provide a national profile of current student assessment practices and institutional support patterns 
within baccalaureate institutions and compare this to assessment practices and support patterns 
found in other types of postsecondary institutions. A second major purpose is to provide advice 
for administrators, faculty, and staff in baccalaureate institutions on designing student assessment 
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approaches and support processes that are likely to foster institutional improvement. Finally, this 
report includes the survey instrument used to gather data for this study in Appendix A. Institutions 
can use this inventory to examine student assessment patterns on their own campuses. 

We will discuss the student assessment methods used by institutions in this report, but it is 
not our intent to examine the specific instruments used or the measurement issues associated with 
their use. For a detailed literature review on student assessment, please consult Improving 
Organizational and Administrative Support for Student Assessment: A Review of the Research 
Literature (Peterson, Einarson, Trice, & Nichols, 1997). The data from our survey research have 
been summarized in several forms including numerous conference papers and journal articles, and 
a technical report entitled Institutional Support for Student Assessment: Methodology and Results 
of a National Survey (Peterson, Einarson, Augustine, & Vaughan, 1999). Readers who are 
interested in the statistical analyses conducted on the student assessment data should consult the 
technical report. While this report relies on those statistical analyses, it will not provide extensive 
statistical details. 



Definition of Student Assessment 

Our focus in this report is on undergraduate student assessment from an institutional 
perspective. We are interested in what approach institutions use to assess student performance, 
how institutions are organized to promote and support student assessment, and how they use 
student assessment data to improve student and faculty performance. 

In this monograph, we use the term student assessment to refer to activities other than 
traditional end-of-course grading that are used to measure a diverse array of dimensions of student 
performance or development. Institutions may decide to engage in student assessment for a variety 
of reasons or differing purposes. Cognitive, affective, and behavioral dimensions of student 
performance and development may be assessed. Assessment efforts may be directed toward 
students as they enter the institution, during their enrollment, or after they exit from the institution. 
Student assessment may use a variety of methods and may be planned and executed centrally 
within institutions or in a decentralized manner through the actions of individual academic units. 
This report will address all these dimensions of student assessment. 



Preview of the Monograph 

This report is presented in nine sections. Following this introduction, the second section, 
perspectives on student assessment, provides a brief recent history of the student assessment 
movement in general and in baccalaureate institutions specifically. The conceptual framework of 
institutional support for student assessment that was developed in the literature review (Peterson et 
al., 1997) and guided the development of the survey instrument is then presented. Section two 
ends with a brief summary of the methods used to conduct the research on institutional support for 
student assessment. 

Sections three through seven present the results of a national survey on institutional support 
for student assessment. These sections parallel the domains in the framework. In each section, 
information is presented for both baccalaureate institutions and all institutions that responded to the 
study. Section three focuses on the influence of external groups, such as state agencies and 
accreditors, on institutional student assessment patterns. In this section, the influence of states on 
institutional student assessment activity is considered for public institutions only. Section four 
focuses on institutions’ approaches to student assessment. This section is followed by a summary 
of the institution-wide support patterns evident in the study. Section six discusses assessment 
management policies and practices used by institutions to promote or assure the use of student 
assessment information. Section seven reports on the institutional uses and impacts of student 
assessment. 
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Section eight is based on the statistical analyses conducted on the national survey data. It 
presents the findings from analyses of three sets of key relationships examined in the study. The 
first focuses on the influence of external groups on institutional approaches to student assessment 
The second examines the relationship of external influences, institutional context, assessment 
approaches, institution-wide support patterns, and assessment management practices and policies 
to the institutional uses of student assessment data. The third analysis examines the relationship of 
external influences, institutional context, assessment approaches, institution-wide support patterns, 
and assessment management practices and policies on the impacts of engaging in student 
assessment. Section nine presents a summary of our research findings and related 
recommendations for specific assessment activities within baccalaureate institutions. It concludes 
with general guidelines for institutions to use to examine and redesign or plan their student 
assessment process and functions. 





2. PERSPECTIVES ON STUDENT ASSESSMENT IN HIGHER EDUCATION 



An overview of the development of student assessment in higher education 
and in baccalaureate institutions suggests the need for the institutional 
perspective that guides this report. A guiding framework conceptualizes 
seven domains of an institution ’s student assessment strategy: institutional 
context, external influences, approach adopted, institutional support patterns, 
assessment management practices and policies, assessment culture and 
climate, and institutional uses and impacts of student assessment. 

This monograph concurs with the perspective of other scholars of student assessment. We 
agree that at its best, student assessment is not simply an exercise to gather data or respond to 
accountability requirements. Rather, an effective student assessment approach is one that gathers 
information about selected aspects of students' characteristics, achievements, and experiences and 
uses this information to shape institutional policies, processes, and practices in ways that lead to 
improved student performance and institutional functioning (AAHE, 1992; Banta & Associates 
1993; Ewell, 1984, 1987c, 1988b; Jacobi, Astin & Ayala, 1987). 

Growth of Student Assessment 

The assessment of student performance in higher and postsecondary education is not a new 
concept or phenomenon. The first College Board examination designed to assess student learning 
outcomes on a national scale was administered in 1901. The ensuing decades are marked by 
events that reflect a growing concern with assessing college student performance, such as: the 
emergence of a regional accreditation focus on student assessment; the establishment of university- 
based and national testing centers; and the development of broadened taxonomies of student 
outcomes (Resnick & Goulden, 1987; Sims, 1992). However, these earlier developments pale in 
comparison to the emergence in the mid 1980s of student assessment as an important focus of 
educational policy at the national, state, and institutional level — a focus that continues today. 

This heightened interest in student assessment is the result of many broad forces. In the 
larger societal and political arena, concerns about consumer protection, the rising costs of 
education, the training and human resource needs for state and regional economic development, 
and fiscal pressures on state and federal government all have shaped public interest in the 
educational contribution of higher education institutions. At the institutional level, the challenges 
of expansion in the 1950s and 60s, enrollment and financial constraints in the 1970s, and new 
educational demands in the 1980s have shifted the managerial focus of performance by higher 
education institutions from resource adequacy, to efficiency, to effectiveness, to broader concerns 
for academic and institutional quality. Consequently, over the past decade and a half the 
assessment and improvement of student performance has been the focus of much of this discussion 
and of many efforts both external to and within colleges and universities. 

A variety of specific activities at the national level — reports, guidelines, legislation, 
educational goals, and funding — have been credited with providing the initial stimulus for the 
student assessment movement in higher education in the last decade and a half. A series of national 
reports that critically examined the quality of education were published in the mid 1980s. The first, 
A Nation at Risk: The Imperative for Educational Reform (National Commission on Excellence in * 
Education, 1983), prompted calls for reform in elementary and secondaiy education. The 
following year Involvement in Learning: Realizing the Potential of American Higher Education 
(National Institute of Education, 1984) addressed the conditions required for improving the quality 
of undergraduate education. Institutions were given three recommendations: set high expectations 
for student learning, actively involve students in learning, and develop an institutionalized process 
for assessing student learning. Subsequent reports such as To Reclaim a Legacy: A Report on the 
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Humanities in Higher Education (Bennett, 1984) and Integrity in the College Curriculum: A 
Report to the Academic Community (Association of American Colleges, 1985) continued this 
focus on issues of undergraduate education quality and assessment. In 1988 the U.S. Department 
of Education revised its Criteria for Recognition of Accrediting Agencies, stipulating that agencies 
must require educational institutions and programs to (1) clearly specify their educational objectives 
and (2) conduct student assessment to determine whether they are achieving these standards. In 
addition, the enactment of legislation such as the "Student Right to Know and Campus Security 
Act" and "Ability to Benefit" legislation (Education Commission of the States, 1991), adoption of 
the National Education Goals (Lenth, 1993, 1996; Nettles, 1995), and provision of federal funds 
for the development of institutions' assessment projects (Cook, 1989) contributed to the early 
momentum of the assessment movement. 

State governments, reflecting these national developments, have also played a key role in 
stimulating postsecondary student assessment. The first state-level mandate for student assessment 
appeared when the Florida state legislature directed the higher education system to develop the 
College-Level Academic Skills Test in 1982. Since then, the number of states involved in student 
assessment has increased steadily with all but four of fifty states now reporting some type of 
student assessment initiative (Cole, Nettles, & Sharp, 1997). States have varied greatly in their 
approaches to student assessment policy, but there is evidence of a general shift in state-level 
approaches over the past decade. From the mid to late 1980s state assessment initiatives generally 
emphasized institutional improvement as their primary purpose and largely permitted institutions to 
design their own assessment efforts. Since 1990, states have placed greater emphasis on student 
assessment as a means of responding to external demands for accountability and have increasingly 
elected to mandate the content and form of institutions' assessment approaches. 

Regional accreditation associations also have emerged as an important influence on student 
assessment. Regional accrediting associations vary in the length of time they have been involved 
in student assessment, their specific reporting requirements, and the range of assessment-related 
services provided to member institutions (Cole et al., 1997). Since the federal government revised 
its criteria for recognizing accrediting agencies in 1988, all six regional accreditation associations 
have required member institutions to undertake and document some form of student assessment 
activity. In turn, a growing number of institutions have reported that accreditation requirements are 
an important reason for deciding to engage in and increase their student assessment efforts (El- 
Khawas, 1995; Johnson et al., 1991). 

Since the mid 1980s the number of postsecondary institutions engaged in some form of 
student assessment activity has steadily increased (El-Khawas, 1988, 1990, 1995). However, by 
1990 only a small proportion of institutions had embarked on comprehensive student assessment 
programs (El-Khawas, 1990; Hexter & Lippincott, 1990; Johnson et al., 1991). Fewer still have 
reported achieving observable impacts from their student assessment efforts (Astin, 1991; 

Hutchings & Marchese, 1990). So it seems that many colleges and universities are investing 
faculty and administrative time and effort in student assessment, but few are benefiting from its 
potential to improve student and institutional performance. 

Characteristics of Baccalaureate Institutions 

Institutional type is an important dimension to consider when examining the approach an 
institution takes toward student assessment. To date,_there has been little examination of 
differences in effective organizational and administrative practices for promoting student 
assessment within specific types of institutions. Yet, effective strategies and practices for 
planning, promoting, and implementing undergraduate student assessment vary according to the 
type of postsecondary institution being considered — for example, what works in promoting 
student assessment in baccalaureate institutions will be different from what works in research 
institutions. In our national study of institutional support for student assessment, we found that 
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there were many statistically significant differences by institutional type on the approaches to, 
support for, practices and policies regarding, and uses and impacts of student assessment reported 
by institutions, and that they were typically stronger than differences by other institutional 
characteristics such as size or whether the institution is private or public. 

The challenges baccalaureate institutions face as they develop student assessment programs 
are different from those faced by other institutions. These differences stem, in part, from the 
institutional mission, curricular focus, governance structure, faculty roles and responsibilities, and 
the student body typically associated with baccalaureate colleges. 

The missions of baccalaureate institutions typically emphasize undergraduate education and 
teaching quality. Therefore, researchers have both speculated and found that student assessment 
activity is more valued and thus more prevalent in baccalaureate institutions than it is in other 
institutional types (Muffo, 1992; El-Khawas, 1993). In fact, our research found that baccalaureate 
institutions do collect more assessment data, at a statistically significant level, than do either 
associate of arts institutions or research universities (Peterson et al., 1999). 

The curriculum at baccalaureate institutions typically focuses on a broad, liberal arts 
education. Despite the similarities of mission and curriculum among baccalaureate institutions, 
they tend to vary greatly by prestige. While many of these institutions are experiencing strong, 
steady enrollment, some have seen declining interest in their institution, perhaps due to students’ 
desires for more vocationally-oriented educational experiences. In these cases, demonstrating 
through student assessment that students succeed both during college and after they graduate could 
become important tools for attracting new students. In fact, some of the leading institutions in the 
assessment movement are baccalaureate institutions (e.g., Alvemo and King’s College). 

Baccalaureate institutions are typically among the least organizationally complex 
postsecondary institutions, in terms of the number of academic departments and structural units 
(Dill, 1997). The generally small institutional size and relative lack of organizational complexity 
should help make the design and implementation of an assessment program more manageable. 
However, because of the smaller size and less complex organization, the goals of both the faculty 
and the administration can drive assessment activities at baccalaureate institution; there may be less 
variance in assessment approaches at these institutions as discrete programs or units may be more 
tightly-coupled to the administration and to other units. At the more highly selective institutions, 
faculty are more likely to share power with administrators in academic decision making than are 
faculty at other baccalaureate institutions (Clark, 1987) and may therefore have more influence on 
assessment activities. A lack of autonomy at the less prestigious institutions suggests that 
administrators may have more influence over decisions to assess students. 

Faculty characteristics at baccalaureate institutions may vary, also, depending on the 
prestige of the institution (Dill, 1997). Faculty at the more prestigious baccalaureate institutions 
tend to engage in more research and may focus less on teaching than do faculty at less prestigious 
institutions. Faculty who focus on research may be less interested in assessing student learning. 

On the other hand, faculty who focus on teaching may be less willing to engage in student 
assessment, as they typically have more limited research experience and expertise. Familiarity with 
conducting research is important in developing both comfort and skill in assessing students. 

Students at the more elite baccalaureate institutions tend to perform quite well. In these 
cases, faculty may wonder if there is a need to assess them. On the other hand, less selective 
institutions tend to enroll students with varying academic abilities. In these institutions, student 
assessment may be highly valued due to both the varied student academic abilities and the focus on 
teaching. 



Although prestige may be a mitigating factor in studying student assessment at 
baccalaureate institutions, we will not present results by prestige in this report. However, we 
received almost four times the number of surveys from “baccalaureate two” institutions (Carnegie, 
1994) than we did from “baccalaureate one” institutions, indicating that our sample of baccalaureate 
institutions is well represented by the less selective institutions. This response rate is not 
surprising, given that in the general population of baccalaureate institutions, there are three times 
more baccalaureate colleges in the second category than in the first. Although the existing literature 
includes some single-institution and other limited institutional case studies of student assessment, it 
does not describe the current assessment activities conducted throughout the country by 
baccalaureate institutions. 

A Framework of Institutional Support for Student Assessment 

An institution’s organizational and administrative context can be a powerful source of 
influence on administrators’, faculty members' and students' behaviors and perceptions, shaping 
their views of what the institution values and which activities are important to engage in. Colleges 
and universities purposefully create a variety of structures, processes, policies and practices to 
support and implement student assessment. Institutions have varied greatly in their approaches to 
assessing student performance and the ways in which they have organized to support their 
assessment efforts, a finding that is hardly surprising given the great diversity and relative 
autonomy of colleges and universities in the American postsecondary system. Yet there is little 
systematic evidence available concerning the specific manner in which postsecondary institutions 
have organized to support student assessment and the differential results of these efforts. For these 
reasons, we have chosen to examine student assessment from a research-based and institutional 
perspective. 

An extensive literature review identified what is currently known about the organizational 
and administrative context for student assessment in postsecondary institutions. A detailed 
description of the literature review process and findings is available in other publications (Peterson 
et al., 1999; Peterson et al., 1997). From this review, we developed a framework of institutional 
support for student assessment. This framework, displayed in Figure 1, consists of seven 
interacting domains. It conceptualizes how institutions respond to external pressures for student 
assessment, how they approach student assessment, the organizational and administrative patterns 
they use to support student assessment, what assessment practices and policies they have adopted 
to promote student assessment, their culture and climate for student assessment, and how student 
assessment information is used by and has impacts on institutions. A seventh domain, institutional 
context, reflects the fact that these patterns probably differ significantly by institutional type, size, 
and whether the institution is public or private. The culture and climate domain is not examined in 
this report and is the focus of future research activity. The other domains are briefly described 
below. A complete listing of the variables included in each domain is provided in Appendix B. 

External Influences on Student Assessment . A variety of external constituencies have 
played an important role in initiating and shaping student assessment efforts within postsecondary 
institutions. In particular, state-level initiatives and regional accreditation associations may exert 
strong direct influences through their requirements for student assessment. The business 
community, private foundations, and professional higher education associations may serve as 
weaker sources of influence — whether as participants in institutions' assessment efforts, as 
consultants or sources of resource materials, or as providers of funds for assessment-related 
activities: National efforts appear to have largely played an indirect role in influencing institutions' 
assessment efforts. 

Institutional Appr oach to Student Assessment . Institutional approach to student assessment 
refers to institutions’ decisions regarding the collection and analysis of student assessment 
information. Important dimensions along which student assessment approaches can be 
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Figure 1. Framework of Institutional Support for Student Assessment 
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differentiated include: the type or content of student assessment measures the extent or array of 
student assessment data collected; the instruments and methods used to collect student assessment 
data; and the analyses conducted and reported for collected data. 

O rg a nizational and Administrative Support for Student Asspssmpnt The literature on 
student assessment identifies five important dimensions of organizational or administrative 
behavior patterns that may support student assessment efforts and enhance the likelihood that 
assessment will contribute to improvements in institutional performance: the institution-wide 
support strategy for student assessment; institution-wide administrative and governance activities 
used to promote student assessment; leadership and faculty support for student assessment; 
planning and coordination of student assessment; and procedures used to evaluate and revise 
student assessment efforts. 

Assessment Management P ractices and Policies . Assessment management practices and 
policies refer to specific practices, policies, procedures, or activities intentionally devised by 
institutions to implement and support their student assessment efforts. A number of content 
activity areas of this domain have been identified including academic resource allocation; student 
information systems; internal access to student information; distribution of assessment reports and 
studies, student involvement in assessment; professional development for student assessment; 
training for student affairs staff; faculty evaluation and rewards; and academic planning and review 
processes. 

Institutional Uses and Impa c ts of Student Assessment . Institutions can use the information 
collected through student assessment processes to contribute to improvement in institutional and 
student performance. Three broad domains of institutional uses and impacts of student assessment 
information are suggested in the literature. These are the use of assessment information for 
institutional decision making, its impact on various internal constituents and phenomena, and its 
impact on the institution’s relationships with its external constituents. 

Institutions may use assessment information within several areas of academic decision 
making such as academic planning and review academic mission and goals, resource allocation and 
faculty evaluation and rewards. Student assessment information may have both internal impacts, 
such as stimulating faculty interest in teaching, and external impacts, such as influencing state 
funding or re-accreditation decisions. The primary concern of our framework is to examine the 
relationship of external influences, institutions’ assessment approaches, patterns of organizational 
and administrative support for assessment, and assessment management practices and policies with 
institutional uses and impacts of assessment information. 

Institutional Context. Broad institutional characteristics such as institutional type, size, and 
whether the institution is public or private are expected to moderate external influences on 
assessment, the institutional approach to student assessment, organizational and administrative 
support patterns, assessment management practices and policies, and institutional uses and impacts 
of assessment information. 



Description of National Survey 

Based on our review of the student assessment literature, we developed a survey 
instrument or inventory called "Institutional Support for Student Assessment" (ISSA). The 
instrument focuses on the assessment of undergraduate students conducted by postsecondary 
institutions — not individual faculty or academic sub-units within institutions. The instrument is 
designed as a comprehensive inventory of external influences on institutions' student assessment 
efforts, institutional approaches to student assessment, organizational and administrative support 
patterns for student assessment, assessment management practices and policies used, and 
institutional uses and impacts of student assessment information. The instrument was intended to 
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assist institutions in obtaining a clear picture of their own student assessment efforts, to provide a 
national profile of undergraduate student assessment efforts in postsecondary institutions, and to 
increase current understanding of how institutions can engage in and promote student assessment 
that produces positive impacts on academic, student and institutional performance. A copy of the 
survey instrument is included in Appendix A. 

In January 1998, the ISSA instrument was mailed to the chief academic administrator at all 
postsecondary institutions recognized by the U.S. Office of Education that offer undergraduate 
programs at the associate or baccalaureate degree level, excluding proprietary and specialized 
institutions. Completed surveys were received from 1,393 of the 2,524 institutions meeting these 
criteria for an overall response rate of 55%. Baccalaureate institutions were well represented in the 
survey responses. We received a total of 316 surveys from baccalaureate institutions - more than 
half of the baccalaureate institutions (53%) to whom instruments were mailed chose to complete 
and return the survey. In addition, more than half of eligible public and private baccalaureate 
institutions participated, and survey responses were quite evenly distributed across states and 
accrediting regions. Given this strong and diversified response rate, we are able to offer a 
representative profile of undergraduate student assessment activities undertaken in baccalaureate 
institutions. Throughout this report we compare student assessment activities within baccalaureate 
institutions to student assessment activities within all responding institutions, including 
baccalaureate institutions. 
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3. THE INFLUENCE OF EXTERNAL GROUPS 



Examines state and regional accreditation association requirements and 
external sources of support for student assessment. While both states and 
regional accrediting agencies have influenced institutions to engage in student 
assessment, accrediting associations are reported to have greater influence 
than states on baccalaureate institutions ’ assessment initiatives. 

State officials and regional accreditation associations have increasingly required 
postsecondary institutions to become engaged in student assessment. Together with professional 
associations and private foundations, they have also offered various services to support 
institutions’ student assessment efforts. There has been relatively little examination of institutions’ 
perceptions of and experiences with these external groups. To examine this domain, our survey 
included questions concerning: state requirements for student assessment; regional accreditation 
association requirements for student assessment; and external sources of support for student 
assessment. The following sections address each of these areas in turn. In considering the state 
role in student assessment, we will focus only on public baccalaureate institutions. We will 
consider all baccalaureate institutions in the examination of the regional accrediting role in student 
assessment. 



State Role in Student Assessment 

States have varied widely in the development and content of their initiatives for 
postsecondary student assessment (Cole et al., 1997). To examine the role of state influences in 
some detail, the survey asked for institutions’ perceptions of three dimensions of state assessment 
plans: the development process and reporting requirements of state assessment plans; the influence 
of state requirements on institutions’ assessment efforts; and state review of institutions’ 
assessment plans or processes. These questions were directed to state-funded institutions only. 

Development and Reporting Requirements of State Assessment Plans 

Development. Scholars have suggested that the influence of state assessment initiatives on 
institutions’ assessment efforts may depend on whether or not institutions have had input in the 
development of the initiatives. Survey respondents were asked to indicate whether their state’s 
plan for student assessment was developed primarily by state officials, through joint consultation 
between state officials and institutional representatives, or whether no state plan or requirement for 
student assessment existed. Responses from baccalaureate and all institutions receiving state 
funding are displayed in Figure 2. 

The majority of baccalaureate institutions receiving state funding report that there is no state 
plan for student assessment (71%). Of the remainder who say there is a state plan, most report that 
it was developed jointly by both state officials and campus representatives (24%). Only 5% report 
that the state plan for student assessment was developed solely by state officials. Compared to all 
institutions, baccalaureate institutions are much more likely to report that there is no state plan for 
assessment. 

Reporti ng Requirements . Scholars also contend that the influence of state assessment 
initiatives will vary with the specific nature of the reporting requirements associated with these 
initiatives. What types of information are institutions required to report to state officials? We 
asked institutions with state plans for assessment which of the following types of information they 
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Figure 2. Development of State Plan for Student Assessment 




state developed jointly no state plan 

plan developed plan 



are required to report as part of their state’s assessment plan: evidence of a student assessment 
plan; measurement of state-mandated student performance indicators; measurement of 
institutionally-developed student performance indicators; and evidence of having used student 
assessment information. Institutions checked all applicable reporting requirements. Figure 3 
presents these responses for public baccalaureate and all public institutions in our study. 

Figure 3. State Reporting Requirements for Student Assessment* 




data 



*Only institutions with a state plan for student assessment responded to this question. 



The most common reporting requirements for baccalaureate institutions are evidence of a 
student assessment plan (88%) and evidence of institutional use of student assessment (74%). 

More than half ofresppndents say they must report on institutionally-devised student performance 
indicators (59%) and are required to" provide evidence of having used student assessment 
information (56%). Compared to all institutions, baccalaureate institutions are more likely to report 
that they must comply with three of these four state requirements. The only requirement which 
appears to be less applicable is requiring measurement of state-mandated student performance 
indicators. 
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Influence of State Assessment Requirements 

What influence have state assessment requirements had on institutions’ undergraduate 
student assessment activities? Institutions were asked which of the following impacts on their 
student assessment activities they attribute to state assessment requirements: were an important 
reason for institution initiating undergraduate student assessment; increased institution’s 
involvement in undergraduate student assessment; were not a factor in institution’s undergraduate 
student assessment activities; or were a negative influence on institution’s undergraduate student 
assessment activities. Institutions selected all applicable influences. Responses from public 
baccalaureate and all public institutions are displayed in Figure 4. 

F*8ure 4. Influence of State Requirements for Student Assessment on Institutions’ 
Assessment Activities* 



*Only institutions with a state plan for student assessment responded to this question 



Among public baccalaureate institutions reporting a state plan, most report that their state’s 
assessment plan has not been a factor in their involvement in student assessment (47%). 

However, a comparable percentage report that the state’s plan has increased their involvement in 
student assessment (42%), and one-third report that the state plan was an important reason to 
initiate student assessment. No respondents report that their state’s assessment mandate has a 
negative influence on their efforts. Compared to all public institutions, public baccalaureate 
institutions are more likely to report that their state’s mandate has not affected involvement in 
student assessment efforts on their campus. 

State Review of Institutions’ Student Assessment Plans 

Occurrence of State Review . States vary in terms of whether and how they evaluate 
institutions’ student assessment plans or information (National Center for Higher Education 
Management System's, 1996). The survey asked respondents whether state officials had reviewed 
or evaluated their institution’s student assessment plans or processes. If a state review or 
evaluation had occurred, respondents indicated whether this review had been conducted by state 
officials, external reviewers, or the institution itself. Responses from public baccalaureate and all 
public institutions are presented in Figure 5. 

Half of public baccalaureate institutions in states with mandated student assessment 
activities report that their student assessment plan or policy has been reviewed. The most common 
form of review is by the state itself (45%). Approximately one-quarter of baccalaureate institutions 
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report that they have been required to do a self-review and the least common method for reviewing 
the institution’s student assessment policy or plan is by external reviewers (8%). Compared to all 
public institutions, public baccalaureate institutions are less likely to have had an external review of 
their assessment plan or process. 

Figure 5. Methods Used to Review Institutions’ Student Assessment Plans or Processes* 
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*OnIy institutions with a state plan for student assessment responded to this question 



State Review Criteria . Institutions that have undergone a state review of their student 
assessment plan or process specified if the evaluation: reviewed the institution’s student 
assessment process; compared the institution’s student performance record to its past performance; 
compared the institution’s student performance record with that of peer institutions; or compared 
the institution’s student performance record with that of other institutions in the state. Responses 
from public baccalaureate and all public institutions are presented in Figure 6. 

Figure 6. State Review Criteria for Institutions’ Student Assessment Plans or Processes 
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For those baccalaureate institutions that had undergone a state-level review of their student 
assessment plan or process, most report that the state reviewed the institution’s student assessment 
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process itself (76%). States also used comparisons in evaluating institutions’ student assessment 
processes. Comparisons with an institution’s own students’ past performance is most common 
(43%), followed by comparisons with other institutions in the same state (29%), and comparisons 
to peer institutions (14%). Compared to all institutions, baccalaureate institutions are less likely to 
be compared with peers. 

Regional Accrediting Role in Student Assessment 

As noted earlier, regional accreditation agencies have been reported as important influences 
on institutions’ decisions to begin or expand their student assessment activities (El-Khawas, 1990, 
1992, 1995; Johnson et al., 1991). The majority of baccalaureate institutions participating in our ’ 
study (81%) had completed a regional accreditation review requiring undergraduate student 
assessment. To further examine the role of regional accreditation associations in student 
assessment, the survey asked about the reporting requirements for regional accreditation and 
institutions’ perceptions of the influence of regional accreditation requirements on their assessment 
efforts. In the following sections, we will present data on the role of the regional accreditation 
body in baccalaureate and all institutions in our study. 

Regional Accreditation Reporting Requirements for Student Assessment 

Regional accreditation associations vary in terms of the evidence they require institutions to 
report on student assessment efforts. It may be expected that requirements to report on the use 
being made of assessment information will contribute to greater institutional impacts from 
assessment than requirements that ask merely for evidence of an assessment plan or assessment 
data. We asked institutions which of the following types of information they were required to 
report to their regional accreditation association: evidence of a student assessment plan; intended 
institutional uses of student assessment information; results of student assessment; and evidence of 
having used student assessment information. Institutions checked all applicable reporting 
requirements. Very few (4%) of baccalaureate institutions were unfamiliar with their regional 
accreditation requirements for student assessment. Responses from baccalaureate and all 
institutions are displayed in Figure 7. 

Figure 7. Regional Accreditation Association Reporting Requirements for Student 
Assessment 
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Most baccalaureate respondents report that they are required to submit all four of these 
criteria to their regional accreditation agency. In almost all cases, they are required to submit 
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evidence that they have a student assessment plan in place (92%). The requirement to submit 
evidence of actual institutional use of student assessment information (79%) is more common than 
the requirement to submit only intended institutional uses (75%). While more than half of 
baccalaureate institutions are required to submit the results of student assessment (66%), this 
requirement is least common among the four. There are no great differences between baccalaureate 
and all institutions on these reporting requirements. 

Influence of Regional Accreditation Association Requirements for Student Assessment 

What influence have regional accreditation association requirements had on institutions’ 
undergraduate student assessment activities? Institutions were asked which of the following 
impacts on their student assessment activities they attributed to their regional accreditor’s 
assessment requirements: were an important reason for initiating undergraduate student 
assessment; increased institution’s involvement in undergraduate student assessment; were not a 
factor in institution’s undergraduate student assessment activities; or were a negative influence on 
institution’s undergraduate student assessment activities. Institutions selected all applicable 
influences. Responses from baccalaureate and all institutions are displayed in Figure 8. 

Figure 8. Influence of Regional Accreditation Requirements for Student Assessment on 
Institutions 9 Assessment Activities 
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Most responding baccalaureate institutions report that regional accreditation requirements 
have increased their involvement with student assessment (84%). Many say that the requirements 
were an important reason to initiate a student assessment process (65%). Very few baccalaureate 
institutional respondents report that the regional accreditation requirements either are not a factor in 
(8%) or have a negative influence on (0.3%) their student assessment process. Regional 
accrediting associations may have a slightly greater effect on baccalaureate institutions than they do 
on all institutions. 

External Sources of Support for Student Assessment 

External constituents may influence institutions’ involvement in student assessment by 
providing funding or services to support assessment efforts. Little is knownabout the extent to 
which institutions have availed themselves of these external sources of support. Our survey asked 
institutions about their use of external grants and other services for improving their student 
assessment efforts. 
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Receipt of External Grants for Student Assessment 

Institutions were asked if they had received grants to improve or support their student 
assessment practices from any of the following external sources: Fund for the Improvement of 
Postsecondary Education (FTPSE), other federal agencies, state incentive programs, and private 
foundations or corporate sources. Responses from baccalaureate and all institutions are presented 
in Figure 9. 

Figure 9. Receipt of External Grants for Student Assessment by Source 
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Approximately one-fifth of responding baccalaureate institutions have received an external 
grant for student assessment. Of the grants that were awarded to baccalaureate institutions, most 
report receiving a private foundation or corporation grant (11%) followed by a FIPSE grant (8%), 
some type of federal grant other than FIPSE (5%), and state grants (3%). Compared to all 
institutions, baccalaureate institutions are more likely to receive private foundation or corporate 
grants to assess students. 

Use of External Services to Support Student Assessment 

A variety of postsecondaiy organizations — professional associations, regional accrediting 
associations, state-level agencies, and consortia of institutions — provide a range of services 
intended to support institutions’ student assessment efforts. These services include consultation, 
assessment conferences, training workshops, and publications or research reports on student 
assessment. For each of these four types of postsecondary organizations, the survey asked 
institutions which, if any, of these student assessment services they had used. Responses from 
baccalaureate and all institutions in our study are presented in Table 1 . 

While many baccalaureate institutions report' that external services are not used or not 
available, they do use assessment-related services from some types of postsecondary organizations 
more than others. They are more likely to use services from professional and regional accreditation 
associations than from state agencies or institutional consortia. They are also likely to use certain 
types of assessment support services more than others, regardless of service provider. 
Baccalaureate institutions make most frequent use of publications or research reports and 
conferences on student assessment. A smaller proportion has used training workshops from one 
or more providers while the smallest proportion has used consultation services. Compared to all 
institutions in our study, baccalaureate institutions are less likely to use state-provided services, 
especially state-sponsored conferences on student assessment. 
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Table 1. Institutional Use of External Services to Support Student Assessment by Service 
Provider 
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Summary Observations 

While both states and regional accrediting agencies have influenced baccalaureate 
institutions to engage in student assessment, accrediting associations are reported to have a greater 
influence than states in increasing involvement in student assessment. Even public baccalaureate 
institutions are likely to report 1) that the state is not a factor in their assessment activities; 2) that 
there has not been a state review of assessment activities; and 3) if there has been a review, it was 
not likely to include state-mandated indicators of student success as one of the criteria. 
Furthermore, baccalaureate institutions are not likely to use state-provided services to support 
student assessment. Compared to all institutions, baccalaureate institutions are less likely to be 
involved with their state on student assessment initiatives. In contrast to state influences, regional 
accreditation agencies are reported to have increased involvement in student assessment in 
baccalaureate institutions. 

Somewhat surprising is the number of baccalaureate institutions that have undergone a 
review of their student assessment plan or process. A majority of institutions have gone through 
an accreditation review and half have experienced a state review of their assessment process. The 
student assessment movement has advanced to the point where almost every institution should 
have had its student assessment process reviewed by at least one external body. These review 
processes go beyond merely reviewing the existence of an assessment plan. For example, 
accreditation bodies seek evidence that student assessment information is used in decision making. 

Despite the apparent influence of external bodies, institutions appear to be using their own 
resources to support their assessment activities. Baccalaureate institutions are not likely to be 
receiving external funding to support their student assessment endeavors. Perhaps funding agents 
believe that assessing students should be a function of the institutional budget, rather than a special 
grant-funded project. 
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4. APPROACHES TO STUDENT ASSESSMENT 



Addresses the type or content of student assessment measures used by 
institutions, the timing of those assessments, their use of standardized 
instruments and less traditional student assessment methods, and types of 
assessment studies conducted and student performance reports produced. 
Compared to all institutions, baccalaureate institutions have more extensive 
and comprehensive approaches to student assessment. They tend to focus on 
collecting affective data and are most likely to collect data on students’ basic 
college-readiness skills, their academic intentions, and their academic 
progress than on any other types of student skills or outcomes. Conversely, 
they are least likely to collect data on vocational skills, students’ personal 
growth, and former students ’ civic or social roles. 

Assessment approach decisions may be influenced by external mandates, institutional 
context, and domains of the organizational and administrative environment. An institution’s 
assessment approach may shape the uses and impacts of assessment information. Assessment 
scholars contend that institutions that collect comprehensive student assessment information — 
those that collect data on various dimensions of student performance, at multiple points in time, 
through a variety of assessment methods — and that conduct and report a variety of analyses of 
assessment data are more likely to use and achieve positive impacts from student assessment data. 

What approaches to student assessment have baccalaureate institutions adopted? The ISSA 
instrument addressed the type or content of student assessment measures used by institutions, the 
timing of those assessments, their use of standardized instruments and less traditional student 
assessment methods, and types of assessment studies conducted and student performance reports 
produced. Here we examine the profile of student assessment approaches in baccalaureate 
institutions and compare it to the profile for all types of postsecondary institutions. 

Type and Extent of Student Data Collected 

Data Collected on Current Students 

What types of data do institutions collect from their students? Survey respondents reported 
the extent to which their institution collects ten types of student data for currently enrolled students 
and four measures for former students. Figures 10, 1 1, and 12 display mean scores for the extent 
to which baccalaureate institutions and all institutions collect these data. 

Figure 10. Extent of Cognitive Data Collected on Current Students 
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Cognitive Data. Figure 10 shows the extent to which institutions collect five types of 
cognitive student assessment data: basic college-readiness skills, higher-order skills (e.g., critical 
thinking and problem solving), general education competencies, competence in major field of study 
(discipline- or program- specific knowledge), and vocational or professional skills. The latter four 
of these five types of data factored together to create a “cognitive assessment” index that we used in 
regression analyses presented in section eight. 

Baccalaureate institutions collect the most cognitive data on students’ basic college- 
readiness skills (3.35), collecting these data for “many” students. Conversely, baccalaureate 
institutions tend to collect less data on vocational skills (1.97) and higher-order skills (2.41), 
collecting these data only on “some” students. Compared to all institutions in our study, 
baccalaureate institutions are more likely to collect data on higher-order skills, general education 
skills, and competence in the students’ major. They are less likely to collect data on basic and 
Vocational skills. 



Affective and Behavioral Data . Figure 1 1 presents five types of affective and behavioral 
data collected on currently enrolled students: personal growth and affective development, student 
experiences and involvement with the institution, student satisfaction with the institution, students’ 
academic intentions, and students’ academic progress. The first three of these items factored 
together into an “affective assessment” index that we used in regression analyses presented in 
section eight. 



Figure 11. Extent of Affective and Behavioral Data Collected on Current Students 
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l=not collected; 2=collected for some students; 3=collected for many students; 4=collected for all students. 



Baccalaureate institutions collect the most data on students’ academic progress (3.87). 
Baccalaureate institutions measure students’ academic progress more frequently than they measure 
students’ academic intentions. One would think that it is necessary to first discern students’ 
academic intentions before evaluating their progress. Nonetheless, it is probably easier to monitor 
academic progress via transcripts and other institutional data than it is to collect and update 
information on students’ goals. Baccalaureate institutions are least likely to collect data on 
students’ personal and affective growth (2.51). However, compared to all -institutions, 
baccalaureate institutions are more likely to collect data on students in all five of these affective and 
behavioral areas. 
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Data Collected on Former Students 



Figure 12 displays mean scores for the extent to which baccalaureate and all institutions 
collect four types of data from former students: civic or social roles (e.g., political, social or 
community involvement), satisfaction and experiences with the institution after leaving, further 
education (e.g., transfer, degree attainment, graduate study), and vocational or professional 
outcomes (e.g., job attainment or performance). These latter three items factored together to create 
a “post-college” assessment index that we used in regression analyses in section eight. 

Figure 12. The Extent of Data Collected on Former Students 
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l=not collected; 2=collected for some students; 3=collected for many students; 4=collected for all students. 



Baccalaureate institutions collect data on “some” to “many” students on vocational 
outcomes (2.74), further education (2.74), and their satisfaction (2.63). They are slightly less 
likely to collect data on students’ post-college civic or social roles (2.26). Compared to all 
institutions in our study, baccalaureate institutions are just as likely to collect post-college data in 
general, and much more likely to collect data on former students’ civic and social roles. 
Baccalaureate institutions often stress that they prepare students for citizenship and social 
responsibility, thus it is fitting that they would report to us that they are measuring these qualities in 
their former students. 



Timing of Data Collection 

Researchers have stressed that it is particularly important for institutions to collect student 
assessment data at varying points in time, in order to assure that students are meeting their 
academic goals and that the college experience is “value-added.” When are baccalaureate 
institutions collecting student assessment data? Table 2 presents the timing of the student 
assessment data collection efforts for each of nine measures of student performance for 
baccalaureate and all institutions. Institutional respondents told us whether they collect such data at 
entry, during student enrollment, and/or at exit. Respondents could choose all three of these 
options. 
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Table 2. Institutions Collecting Cognitive, Behavioral and Affective Student. Assessment 
Data at Entry, While Student is Enrolled and at Exit 

■ % Institutions Collecting Data 



At Entry While Enrolled At Exit 



Data Collected 


Bacc 


All 


Bacc 


All 


Bacc 


All 


Cognitive Data 


Basic Skills 


94 


97 


25 


21 


10 


8 


Higher Order Skills 


41 


32 


64 


66 


34 


33 


General Education 


37 


32 


68 


64 


36 


36 


Major Competence 


8 


9 


58 


62 


68 


64 


Vocational Skills 
Affective and Behavioral 


9 


10 


72 


67 


49 


60 


Data 


Personal Growth 


44 


32 


59 


67 


48 


41 


Student Involvement 


10 


7 


71 


72 


58 


55 


Student Satisfaction 


9 


7 


70 


71 


67 


65 


Academic Intentions 


93 


92 


37 


36 


28 


19 



Timing of Collecting Cognitive Data 

With the exception of basic skills data, most cognitive data are collected either while the 
student is enrolled or at exit. It is not surprising that most baccalaureate institutions which collect 
basic skills data do so at the point of entry into the institution (94%). However, only one-quarter 
of these institutions report that basic skills are assessed while the student is enrolled. More than 
half of those baccalaureate institutions that collect data on a student’s competence in the major do 
so while the student is enrolled (58%) and again at exit (68%). While this finding seems to 
indicate that students are assessed at more than one point in time, in actuality, only 27% of 
baccalaureate institutions are collecting major competence data on students both while enrolled and 
at exit— the remaining institutions are collecting major competence data either while enrolled or at 
exit. Compared to all institutions, baccalaureate institutions are less likely to collection vocational 
data on students at exit — these institutions probably have fewer vocationally-oriented programs. 

Timing of Collecting Affective and Behavioral Data 

With the exception of academic intentions, most affective data are collected either during 
enrollment or at exit. In terms of academic intentions, it again is not surprising that students’ 
intentions are collected at entry (93% collect these data at entry). It seems apparent that these 
intentions are not assessed again, even though students’ goals may change throughout their time at 
an institution. Compared to all institutions, baccalaureate institutions are more likely to collect data 
on students’ personal growth at entry as a baseline for further growth. 

Student Assessment Instruments and Methods Used 

Institutions engaging in student assessment efforts must select the means by which 
assessment data will be collected. A traditional choice is to use assessment instruments, generally 
in the form of objective examinations or inventories administered in a pencil and paper or 
-computerized format. More recently, there has been growing interest in alternative” methods of 
assessing students. These non-traditional methods tend to be more qualitative or integrative in 
approach — requiring students to demonstrate higher-order cognitive skills such as application or 
synthesis — or may use sources of information other than students themselves. Further, as the 
student body entering postsecondary institutions becomes more diverse through the increased 
enrollment of part-time, older, and minority students, some scholars have suggested the need to 
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use special assessment methods to reflect the unique characteristics, learning styles, needs, and life 
situations of various sub-populations of students. 

Student Assessment Instruments 



How are institutions collecting student assessment data? Survey respondents reported 
whether their institution uses instruments or tests to collect any of ten types of assessment 
information: nine for current students (basic college-readiness skills; higher-order skills; general 
education competencies; competence in major field of study; vocational or professional skills; 
academic intentions, personal growth and affective development, experiences and involvement 
with the institution; and satisfaction with the institution) and one for former students (alumni 
satisfaction and experiences). If an instrument is used, respondents specified its source — 
institutionally-developed, state-provided, or commercially available. Figures 13 and 14 present 
information on these ten types of assessment information collected via traditional instruments or 
tests by baccalaureate and all institutions in our study. 



Figure 13. Institutional Use of Student Assessment Instruments to Collect Cognitive Data 




As Figures 13 demonstrates, the use of assessment instruments or tests varies with the type 
of assessment data collected. In terms of cognitive data, baccalaureate institutions are most likely 
to use instruments or inventories to collect data on basic college-readiness skills and competencies 
in the major field (both 88%). These institutions are least likely to use instruments or inventories 
to collect data on students’ higher-order cognitive (58%) and vocational (53%) skills. Compared 
to all institutions, baccalaureate institutions are less likely to use instruments to measure students’ 
basic college-readiness and vocational skills, but are more likely to measure students’ competencies 
in the major using instruments or tests. 

Figure 14 demonstrates that in terms of affective and behavioral data, baccalaureate 
institutions are most likely to use instruments or inventories to=collect data on current student 
(93%) and alumni (91%) satisfaction. They are less likely to use instruments or inventories to 
collect data on students’ personal and affective growth (70%). Compared to all institutions, 
baccalaureate institutions are more likely to use instruments or tests to measure students’ personal 
or affective growth. 
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Figure 14. Institutional Use of Student Assessment Instruments to Collect Affective and 
Behavioral Data 




Other Student Assessment Methods 

Institutions also reported the extent to which they use each of nine alternative methods of 
student assessment. Table 3 presents this information for baccalaureate and all institutions. Four 
of these options require more active participation of currently enrolled students and factored 
together to create a “student-centered” assessment index that we used in regression analyses 
presented in section eight: student performance in capstone courses, observations of student 
performance, student portfolios or comprehensive projects, and student interviews or focus 
groups. Baccalaureate institutions are most likely to use student performance in capstone courses 
(2.50). The other three student-centered methods are used by baccalaureate institutions only in 
some units. The least often used of these three methods is interviews or focus groups with 
students (1.96). Compared to all institutions in our study, baccalaureate institutions are more 
likely to use all four of the student-centered methods for collecting assessment data. 



Table 3. Mean Scores of Extent of Use of Other Student Assessment Methods 



Student Assessment Methods 


Baccalaureates (n=315) 


All Institutions (n=1393) 


Student-Centered 


Student performance in capstone courses 


2.50 


2.15 


Observations of student performance 


2.34 


2.26 


Student portfolios or comprehensive projects 


2.29 


2.10 


Student interviews or focus groups 


1.96 


1.84 


Externally Oriented 


Alumni interviews or focus groups 


1.95 


1.90 


Employer interviews or focus groups 


1.66 


1.87 


Other Methods 






Surveys or interviews with withdrawing students 


' 2.78 


2.40 


Transcript analysis 


. 2.24 


2.16 


External examination of students 


1.98 


2.02 



1 = not used; 2 = used in some units; 3 = used in most units; 4 = used in all units 




24 

33 



Two of the other alternative measures rely on information from external groups or sources 
and factored together to create an index of “external methods” of assessment that we used in 
regression analyses presented in section eight: alumni interviews or focus groups and employer 
interviews or focus groups. Baccalaureate institutions are more likely to interview or hold focus 
groups with alumm (1.95) than with employers (1.66). Compared to all institutions, baccalaureate 
institutions are more likely to interview alumni and less likely to interview employers. 

There were three additional alternative student assessment measures listed in our survey: 
surveying or interviewing withdrawing students, transcript analysis, and using external 
examinations. Of these three, baccalaureate institutions are most likely to survey/interview 
withdrawing students (2.78). They are least likely to use external examiners or examinations of 
students and their use of transcript analysis falls in the middle of these two. Baccalaureate 
institutions are more likely to survey or interview withdrawing students and to use transcript 
analysis than are all institutions. 

Assessment Methods for Student Sub-Populations 

Do institutions use different assessment methods for specific sub-populations of their 
student body ? Table 4 presents the percentage of baccalaureate and all institutions using different 
assessment methods for four student sub-populations: adult students, part-time students, minority 
students, and distance education students. 



Table 4. Institutional Use of Different Assessment Methods for Different Student Sub- 
Populations ■ 



Student Sub-Population 


% Institutions Using Different Assessment Methods 


Baccalaureate (n=306) 


All Institutions (n=1366) 


Adult students 


17 


10 


Part-time students 


7 


5 


Minority students 


1 


2 


Distance education students 


22 


22 



Most baccalaureate institutions do not use different assessment methods for specific sub- 
populations of undergraduate students. However, 22% report using different methods for distance 
education students, a decision that may be necessitated by the type of medium used for course 
delivery. In addition, 17% report using different methods to assess adult students. Very few use 
special methods for collecting assessment data from their part-time (7%) or minority (1%) 
students. Compared to all institutions, baccalaureate institutions are more likely to use different 
assessment measures for adult students. Perhaps adult students are more of an anomaly on these 
campuses, thus making it easier to separate these students from the rest and use special assessment 
techniques for them. 



Student Assessment Studies and Reports 

Beyond collecting descriptive data on their students’ characteristics, performance, or 
development, institutions must consider how to transform these data into useful information. 
Institutions that study the relationship between various aspects of students’ institutional 
- - - experiences and their performance will be better able to make informed decisions concerning 
academic and student-related policies and practices. Another consideration is the level of 
aggregation at which assessment information is analyzed and reported. 
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Student Assessment Studies 



Are institutions studying how student performance is connected to experiences with the 
institution? In our survey, we asked if institutions conduct studies of the relationship between nine 
areas of students’ institutional experiences and students’ performance. Table 5 identifies these 
areas and presents data on institutional studies for both baccalaureate and all institutions in our 
study. 



Table__5 I _Institutions Conducting Student Assessment Studies 



Studies of Relationship Between Student Performance 
and the Following Experiences 


% Institutions Conducting Studies 
Baccalaureates All Institutions 

(n=304) (n=1329) 


Do not study these experiences 


34 


38 


Admission standards or policies 


50 


42 


Extra-curricular activities 


30 


24 


Student financial aid and/or concurrent employment 


27 


30 


Academic advising patterns 


27 


26 


Residence arrangements 


. 27 


21 


Student course-taking patterns 


23 


26 


Classroom, library and/or computing resources 


17 


17 


Exposure to different instructional or teaching methods 


16 


21 


Patterns of student-faculty interaction 


16 


34 



One-third of baccalaureate institutional respondents (34%) report that they do not conduct 
any of these studies on students’ experiences. For those that do conduct studies, they are most 
likely to study relationships with admissions standards or policies (50%). They are least likely to 
conduct studies on classroom, library and/or computing resources (17%), exposure to different 
instructional or teaching methods (16%), and patterns of student-faculty interaction (16%). 

Compared to all institutions, baccalaureate institutions are more likely to study relationships 
between student performance and both extra-curricular activities and the institutional admissions 
standards or policies. Baccalaureate institutions often offer a plethora of extra-curricular activities 
as part of their mission for holistic postsecondary education. In addition, since baccalaureate 
institutions vary in terms of selectivity, studies of admissions standards may be especially relevant 
to these institutions. 

Student Assessment Reports 

Finally, institutions were asked to report the levels of aggregation at which they provide 
reports of student assessment information. Figure 15 shows the percentage of baccalaureate and 
all institutions providing student assessment reports at five levels of aggregation: institution-wide; 
school or college; program; special populations of students; course or groups of courses. 
Respondents indicated as many levels of aggregation as were applicable or could indicate that they 
provide no reports of student assessment information. 

Most baccalaureate respondents prepare some reports on student assessment results (91%). 
The most prevalent level of aggregation used is institution-wide (69%), followed closely by 
academic program/department (64%). The least prevalent level of aggregation for baccalaureate 
institutions is by school or college (19%). Compared to all institutions, baccalaureate institutions 
are less likely to provide student assessment data at the school or college level, likely because these 
institutions are not typically organized into discrete schools or colleges. Baccalaureate institutions 
are also less likely to provide reports either by subgroups of students or by course. 
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Figure 15. Preparation of Student Performance Reports by Level of Aggregation 




Summary Observations 

Compared to all institutions, baccalaureate institutions appear to be taking the lead in terms 
of their approach to student assessment. They tend to focus on collecting affective data and are 
most likely to collect data on students’ basic college-readiness skills, their academic intentions, and 
their academic progress than on any other types of student skills or outcomes. Conversely, they 
are least likely to collect data on vocational skills, students’ personal growth, and former students’ 
civic or social roles. Compared to all institutions, baccalaureate institutions are more likely to use 
extensive methods for collecting data, including a greater emphasis on student-centered methods. 
Finally, two-thirds of all responding baccalaureate institutions said they conduct studies on 
students linking their performance to their interaction with the institution, and a great majority of 
respondents provide reports on the results of student assessment initiatives. 
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5. ORGANIZATIONAL AND ADMINISTRATIVE SUPPORT PATTERNS 

Focuses on the patterns of organizational and administrative support that 
institutions have developed to promote student assessment on their 
campuses, such as their mission and purpose, institution-wide activities, 
leadership support, and planning and coordinating mechanisms. 
Baccalaureate institutions’ patterns reflect strong institution-wide support for 
student assessment and extensive faculty involvement. 

An important focus of our study was to examine the patterns of organizational and 
administrative support institutions have developed to promote student assessment on their 
campuses. The assessment literature suggests that institutions will be most likely to promote 
internal support for assessment if they engage in assessment for internal rather than external 
purposes, have visible and strong leadership support for assessment, include a broad range of 
internal participants — particularly faculty — in making assessment-related decisions, and 
regularly evaluate their assessment programs. It is expected that organizational and administrative 
support will shape the assessment approach, assessment management practices and policies, and 
ultimately, the institutional uses and impacts of assessment information. 

The ISSA instalment included questions concerning the institution-wide assessment 
support strategy; administrative and governance activities used to promote student assessment; 
leadership and faculty support for assessment; planning and coordination for assessment; and 
evaluation of student assessment processes. Once again, we will present information concerning 
die pattern of organizational and administrative support for student assessment in baccalaureate 
institutions and compare it to the pattern for all types of postsecondary institutions. 

Institution-Wide Assessment Support Strategy 

Mission Emphasis 

Assessment scholars and practitioners suggest that institutions’ academic mission content 
may symbolize the importance with which student assessment is regarded and so may influence the 
degree of internal support for assessment. What do the mission statements of our responding 
institutions tell us about their values regarding student assessment? Institutions reported whether 
their mission statement explicitly: a) emphasizes excellence in undergraduate education, b) 
identifies intended student outcomes, c) refers to student assessment as an important priority, or d) 
does not mention any of these. Respondents could choose all applicable responses. Figure 1 6 
displays the percentage of baccalaureate and all institutions reporting each of these mission 
statement components. 

Although few baccalaureate institutions refer to student assessment as an important activity 
in their mission statement ( 1 6%), most baccalaureate institutions emphasize excellence in 
undergraduate education in their mission statements (88%) and many identify intended student 
outcomes (61%). Compared to all institutions, baccalaureate institutions are more likely to 
emphasize excellence in undergraduate education and specify intended student outcomes in their 
mission statements. They are less likely to specifically refer to the importance of student 
assessment. 

Purposes for Engaging in Student Assessment 

Why are institutions engaging in student assessment? A second dimension of institution- 
wide assessment support strategy examined in this study is the purpose(s) underlying institutions’ 
student assessment efforts. Scholars contend that whether assessment is primarily engaged in for 
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Figure 16. Institutional Mission Emphasis 




education outcomes assessment 



internal or for external purposes may influence the nature of an institution’s assessment approach, 
degree of internal support, and assessment uses and impacts. Institutions rated the importance of 
six purposes for engaging in student assessment: preparing for an institutional accreditation self- 
study; meeting state reporting requirements; guiding internal resource allocation decisions; guiding 
undergraduate academic program improvement; improving the achievement of undergraduate 
students; and improving faculty instructional performance. The last four of these purposes were 
factored in an index of internal purposes for student assessment that we used in regression 
analyses presented in section eight. Figure 17 presents the mean scores for each student 
assessment purpose for baccalaureate and all institutions on this question. 



Figure 17. Purposes of Student Assessment 




self-study reports resources programs improvement improvement 

I : internal purposes index — | 

l=no importance; 2=minor importance; 3=moderate importance; 4=very important 



Baccalaureate institutions report that preparing a self-study for accreditation is the most 
important reason to engage in student assessment (3.63), followed by guiding undergraduate 
academic program improvement (3.51), improving the achievement of undergraduate students 
(3.47), and improving faculty instructional performance (3.08). All are rated “moderate” to “very” 
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important. They are less likely to say that improving meeting state reporting requirements (2.30) 
or guiding internal resource allocation decisions (2.62) are important reasons for engaging in 
student assessment. 

Compared to all institutions, baccalaureate institutions are less likely to report that meeting 
state requirements is an important reason to engage in student assessment. A greater percentage of 
baccalaureate institutions are private — likely explaining this difference. A smaller percentage of 
baccalaureate institutions are also likely to say that preparing for a self-study is an important reason 
to engage in student assessment. 

Institution-Wide Administrative and Governance Activities 

Institutions may develop a variety of administrative activities and governance structures to 
promote student assessment on an institution-wide basis. How prevalent are these practices? The 
ISSA instrument asked whether institutions use any of the following administrative activities and 
governance structures: annual institution-wide initiatives on assessment; rewards or incentives for 
academic and student affairs administrators to use student assessment in their units; incentives for 
academic units to use assessment information in evaluation and improvement efforts; student 
assessment workshops for academic and student affairs administrators; board of trustees committee 
that addresses student assessment; faculty governance committee that addresses student assessment 
issues; and student representation on student assessment committees. Figure 18 displays the 
percentage of baccalaureate and all institutions that have introduced these activities or structures in 
their institutions. 

Figure 18. Institution-Wide Administrative and Governance Activities Used to Promote 
Student Assessment 
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Most of these seven activities are used sparingly by baccalaureate institutions. They are 
most likely have a faculty governance committee that addresses assessment issues (69%) and to 
provide assessment workshops for academic and student affairs administrators (49%). They are 
least likely to provide rewards or incentives for academic and student affairs administrators who 
promote units’ use of assessment (6%) or to have a board of trustees committee that addresses 
assessment (14%). 

Compared to all institutions, baccalaureate institutions are more likely to have a faculty 
governance committee that addresses assessment issues. Perhaps faculty at these institutions are 
more likely to discuss teaching and learning issues in group settings. Baccalaureate institutions are 
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less likely to either hold annual institution-wide student assessment forums or to provide 
assessment workshops for academic and student affairs administrators. 

Leadership and Faculty Support for Student Assessment 

The degree to which student assessment efforts are supported internally is likely an 
important determinant of the extent and impact of an institution’s student assessment activities. 
How supportive of student assessment are various internal constituencies? We asked survey 
respondents to rate the degree to which six internal groups support undergraduate student 
assessment activities: board of trustees; chief executive officer; academic affairs administrators; 
student affairs administrators; faculty governance; and students. Figure 19 presents the mean 
responses to this question for baccalaureate and all institutions in our study. 



Figure 19. Internal Constituent Support for Student Assessment 




affairs affairs governance 



l=very unsupportive; 2=somewhat unsupportive; 3=neutral, unknown; 4=somewhat supportive; 5=very supportive 



Baccalaureate respondents report that academic affairs administrators are most supportive 
(4.69), followed by the chief executive officer (4.39) and student affairs administrators (4.33). 
Students are perceived as being the least supportive (3.40). The patterns of support within 
baccalaureate institutions are quite similar to the patterns of support for all institutions. 

Planning and Coordinating Student Assessment 

A central issue in discussions of leadership and governance for student assessment 
concerns the degree to which responsibility for planning and coordinating student assessment 
activities is centralized or decentralized within institutions. Centralized approaches that focus such 
responsibility on senior administrators or central offices may signal that assessment is a valued 
institutional activity, but decentralized approaches involving an array of internal participants may 
do more to promote broader internal support for assessment. How do institutions plan and 
coordinate their assessment efforts? We asked institutional respondents about six aspects of their 
assessment planning and coordination process: the nature of an institutional plan or policy for 
student assessment; membership on an institution-wide planning group for assessment; executive 
responsibility for the assessment planning process; approval authority for assessment plans; 
operational responsibility for student assessment; and reporting patterns. 
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Institutional Plan or Policy for Student Assessment 



We asked survey respondents which of seven types of institutional plans or policies for 
student assessment best describes what exists at their institution: 1) formal comprehensive 
centralization — a formally adopted plan or policy specifying undergraduate student assessment 
activities for all academic programs or units; 2) formal limited centralization — a formally adopted 
plan or policy for undergraduate student assessment in some academic programs or units; 3) formal 
decentralization — a formally adopted institutional plan or policy requiring all academic units or 
programs to develop their own undergraduate student assessment plan; 4) formal guidance — a 
formally adopted institutional plan or policy identifying institution-wide activities to be conducted 
by a central committee or office; 5) informal encouragement — no institutional plan or policy but 
academic units or programs are encouraged to develop their own undergraduate student assessment 
activities; 6) emergent — institution is currently developing a plan or policy for undergraduate 
student assessment; and 7) none — institution does not have an undergraduate student assessment 
plan or policy. Respondents could select more than one type of plan or policy. Figure 20 presents 
the percentage of baccalaureate and all institutions using each type of assessment plan or policy. 



Figure 20. Types of Institutional Plan or Policy for Student Assessment 
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Virtually all baccalaureate institutions report that they have a plan or policy for student 
assessment; only 2% have no policy or plan. Baccalaureate institutions are most likely to report 
that they have a formal centralized student assessment policy (55%), followed by a formal 
decentralized policy (46%), and a formal guidance plan (40%). They are less likely to have any of 
the other types of plans or policies. Compared to all institutions, baccalaureate institutions are just 
as or more likely to have all types of these plan or policy approaches, except for the formal limited 
centralization policy. One could interpret from this finding that all academic units in baccalaureate 
institutions are treated fairly equally — either all are subjected to an assessment policy or none are. 

Institution-Wide Assessment Planning Group Membership 

Who is involved in student assessment planning? We asked respondents whether they 
have an institution-wide group responsible for ongoing planning and policy setting for 
undergraduate assessment. Fully 73% of baccalaureate institutions and 70% of all institutions 
report having such a group. Of those institutions with institution-wide planning groups, we asked 
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which of the following internal constituents serves on the group: chief executive officer, academic 
affairs administrator or staff, student affairs administrator or staff, institutional research 
administrator or staff, academic review and evaluation administrator or staff, student assessment 
administrator or staff, faculty, and students. Respondents checked as many constituents as were 
applicable. Figure 21 presents the percentage of baccalaureate and all institutions that responded 
that each constituent is a member of their assessment planning group. 



Figure 21 . Membership on Institution-Wide Student Assessment Planning Group* 




♦Only institutions with an institution-wide planning group for student assessment responded to this question 



For baccalaureate institutions with a planning group, student assessment planning 
committees are staffed mainly by faculty (in 90% of the institutions) and academic administrators 
(85%). They are least likely to have the chief executive officer (12%), academic review or 
evaluation staff (20%), or student assessment staff (22%) involved. Compared to all institutions, 
baccalaureate institutions are less likely to have all the listed constituents serving on their planning 
group, except for students. 

Executive Responsibility for Assessment Planning Process 

Another measure of the governance process used for student assessment concerns where 
formal leadership for assessment planning is vested in an institution. We asked institutions which 
of the following positions has executive responsibility for or chaire the institution-wide group for 
student assessment planning: academic affairs administrator; student affairs administrator; 
institutional research officer; academic review and evaluation officer; student assessment officer; or 
faculty member. Figure 22 presents this information for baccalaureate and all institutions. 

At most baccalaureate institutions, academic administrators chair the student assessment 
planning group (60%), followed by a faculty member at 28% of baccalaureate institutions. Less 
than 20% of the institutions have an institutional research officer or a student assessment 
administrator chair the student assessment planning group. It is very rare for either academic _ 
review (5%) or student affairs "(3%) administrators to chair the student assessment planning group 
at baccalaureate institutions. The pattern is similar for all institutions. Baccalaureate institutions 
are slightly more likely than are all institutions to appoint an academic affairs administrator to chair 
the student assessment planning group. 



O 

ERIC 



33 



42 



\ 



Figure 22. Executive Responsibility for Student Assessment Planning Group 
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Approval Authority for Student Assessment Plan 

We asked institutions to identify who, among ten possible positions or groups, approves 
changes in their institutional plan or policy for student assessment: board of trustees; chief 
executive officer; chief academic affairs officer; chief student affairs officer; institutional research 
officer; academic review and evaluation officer; student assessment officer; student government; 
academic senate or other faculty committee; and faculty union. All respondents could answer this 
question regardless of whether or not they had an institution-wide committee for assessment 
planning. Respondents indicated as many positions or groups as were applicable. Only 15% of 
baccalaureate institutions report that academic review and evaluation officers, student government, 
or faculty union personnel have approval authority for the institutional assessment plan or policy. 
Figure 23 presents the results for seven positions for baccalaureate and all institutions. 

Figure 23. Approval Authority for Student Assessment Plan or Policy 




academic student assessment senate 

affairs affairs 



ERIC 



34 



43 




Over 70% of baccalaureate institutions report that the chief academic officer has approval 
authority for student assessment. An academic senate has such authority in approximately half the 
institutions and the chief executive officer has such authority in 40% of responding institutions. 
Baccalaureate institutions are not likely to invest approval authority in a student assessment officer 
(9%), the board of trustees (12%), a chief student affairs officer (13%), or an institutional 
researcher (21%). Compared to all institutions, baccalaureate institutions are more likely to invest 
an academic senate with approval authority. Boards at baccalaureate institutions may be more 
involved with teaching and learning issues than are boards at four year institutions. Conversely, 
baccalaureate institutions are less likely to invest the chief academic officer or the academic senate 
with student assessment approval authority. They are less likely to invest the board of trustees, the 
chief executive officer or the chief student affairs officer with student assessment approval 
authority. 

Operational Responsibility for Day-to-Day Student Assessment Activities 

Beyond the planning phase of student assessment, institutions must decide where to place 
responsibility for overseeing the day-to-day undergraduate student assessment activities such as 
instrument development, data collection, analysis, and reporting. Where have institutions placed 
operational responsibility for student assessment? We asked institutions which of six positions or 
offices has responsibility for day-to-day student assessment activities: academic affairs 
administrator; student affairs administrator; institutional research officer; academic review and 
evaluation officer; student assessment officer; and faculty member(s). Institutions checked as 
many positions/offices as applied. Figure 24 presents the results for baccalaureate and all 
institutions. 



Figure 24. Operating Responsibility for Day-to-Day Student Assessment Activities 
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At baccalaureate institutions, an academic affairs administrator (54%), an institutional 
research officer (42%), or a faculty member (27%) is most likely to be responsible for day-to-day 
operational responsibility for student assessment. Less frequently, an academic review officer 
(10%), an assessment officer (13%), or a student affairs administrator (14%), is in charge of day- 
to-day activities.- Compared to all institutions,-baccalaureate institutions are more likely to have an 
academic affairs administrator or a faculty member in charge of day-to-day responsibilities. 
Conversely, baccalaureate institutions are less likely to have a student affairs administrator in 
charge of day-to-day activities. 
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Reporting Relationship of Individual with Operating Responsibility for Student Assessment 

Finally, institutions were asked to whom the individual with day-to-day operating 
responsibility for student assessment reports. Survey respondents selected from the following five 
offices: chief executive officer; chief academic officer; chief student affairs officer; institutional 
research officer; and academic review and evaluation officer. Figure 25 presents the responses 
from baccalaureate and all institutions. 

Figure 25. Reporting Relationship of Individual with Operating Responsibility for Student 
Assessment Activities 
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For baccalaureate institutions, assessment managers responsible for day-to-day activities 
most often report to the chief academic officer (66%) and in some institutions report to the chief 
executive officer (28%). In hardly any institutions do they report to the chief student affairs officer 
(2%), the institutional research officer (2%), or the academic review and evaluation officer (2%). 
Compared to all institutions, assessment managers at baccalaureate institutions are more likely to 
report to the chief academic officer. 

Evaluating the Student Assessment Process 

The student assessment literature insists upon the importance of institutions regularly 
evaluating their assessment processes. Consequently, our survey inquired whether institutions 
have evaluated their student assessment plan or process and if so, what elements of their plan or 
process have been reviewed. 

Status of Evaluation of Student Assessment Plan or Process 

Institutions were asked whether they have conducted a formal evaluation, an informal 
evaluation, are currently developing plans for an evaluation, or are not planning to evaluate their 
assessment process. The results for baccalaureate and all institutions are depicted in Figure 26. 

Approximately half (47%).of all baccalaureate institutions have evaluated their assessment 
plan; 21% have conducted a formal evaluation and 26% have conducted an informal evaluation. If 
they have not yet evaluated their assessment plan, baccalaureate institutions are likely to be 
developing an evaluation plan (33%) and slightly less likely to not be planning to evaluate their 
assessment process (2 1 %). Compared to all institutions, baccalaureate institutions are slightly less 
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likely to have conducted an evaluation and slightly more likely to be planning to conduct an 
evaluation. " 

Figure 26. Status of Student Assessment Evaluation 
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Elements of Student Assessment Evaluation 

Those institutions that have formally or informally evaluated their student assessment 
process were asked which of eight elements were reviewed as part of the evaluation: student 
assessment plans and policies; structure and responsibility for student assessment; achievement of 
intended objectives for student assessment; reliability and validity of assessment instruments and 
methods; quality of data analysis; use of assessment information in institutional decision making; 
problems encountered while conducting assessment; and the costs and benefits of student 
assessment. Results of this question for baccalaureate and all institutions are shown in Figure 27. 



Figure 27. Elements of Student Assessment Evaluation 
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Among the baccalaureate institutions that evaluate their student assessment process, most 
take several elements of that process into consideration. They are most likely to assess then- 
student assessment plan and policies (90%), the achievement of intended objectives for student 
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assessment (76%), the problems encountered while conducting assessment activities (74%), the 
use of assessment information in decision making (73%), and the structure and responsibility for 
student assessment (73%). These institutions are least likely to compare the costs and benefits of 
student assessment (3 1 %). Compared to all institutions, baccalaureate institutions are more likely 
to take all these components into consideration during an evaluation except for the reliability and 
validity of assessment instruments and methods. 

Summary Observations 

Baccalaureate institutions seem to have strong institution-wide support for student 
assessment. The student assessment policies on baccalaureate campuses tend to be applied to all 
academic units across the institution. Although these institutions are not likely to directly refer to 
student assessment in their mission statements, they have a strong focus on undergraduate 
education in general and they are likely to have enunciated specific outcomes for their 
undergraduates in their mission statements. They also report a combination of internal and external 
purposes for engaging in student assessment. 

Furthermore, these institutions tend to involve faculty in the student assessment process. 
Compared to all institutions, academic senates are more likely to have approval authority for 
changes to student assessment plans or policies, there is more likely to be a faculty committee that 
addresses assessment, faculty are more likely to be in charge of the day-to-day assessment 
activities, and in almost all baccalaureate institutions, faculty members are represented on an 
institution-wide committee for student assessment. Students are also more likely to be represented 
on these committees in baccalaureate institutions than they are in all institutions. 

Despite the involvement of students and faculty, administrators are still perceived as more 
supportive of student assessment than are either faculty or students. Academic affairs 
administrators are the student assessment leaders on most baccalaureate campuses and are also 
most likely to manage the day-to-day student assessment responsibilities. Student affairs 
administrators are not as involved with student assessment as they are in all institutions. 
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6. ASSESSMENT MANAGEMENT PRACTICES AND POLICIES 



Reports on the existence of specific institutional practices and policies 
designed to promote student assessment management. Baccalaureate 
institutional administrators seem to be taking an unobtrusive approach in 
setting policies and developing practices — there is very little use of student 
assessment information in making either budget or faculty evaluation and 
reward decisions. However, compared to all institutions, they are more likely 
to reward or evaluate faculty based on assessment participation or outcomes. 
Less obtrusive policies, such as distributing assessment results and providing 
access to student performance information are more prevalent in baccalaureate 
institutions. 



A fourth domain in our conceptual framework is that of assessment management practices 
and policies providing mechanisms for managing the student assessment process and directing the 
ways in which student assessment information is used throughout the institution. Assessment 
management practices and policies are suggested in the literature as powerful means through which 
institutions can support student assessment and encourage the use of collected assessment 
information. Conceptual dimensions of assessment management practices and policies such as 
their comprehensiveness, consistency, and the extent to which they are employed within an 
institution are expected to influence internal support for student assessment and the likelihood of 
achieving institutional impacts from assessment information. 

Our survey asked about 41 specific institutional practices and policies promoting student 
assessment management. These items were factor analyzed to create nine comprehensive indices. 
In this section we will present results of these assessment management practice and policy indices, 
rather than of individual items. 

Practices for Managing Student Assessment 

What kinds of practices do institutions develop to manage and promote their student 
assessment process? We asked respondents about the existence of a variety of specific institutional 
practices to manage the student assessment process in four areas: academic resource allocation, 
student information systems, internal access to student assessment information, and distribution of 
assessment reports and studies. Scores for institutions for all specific practices in these four areas 
are presented in Appendix A. We used factor analysis to create indices of management practices in 
these four areas. Briefly, the four indices are: 

1. Academic Budget Decisions: whether the institution’s academic budget process compares 
academic units on student performance indicators and allocates resources competitively among 
them; or rewards them for improvement based on student performance indicators. 

2. Computer Support: whether the institution has a computerized student information system that 
includes student performance indicators; tracks students from application through graduation; and 
is integrated with faculty, curricular, and financial databases. 

3. Access to Student Information: whether assessment information on individual students is 
available to institutional researchers, assessment or evaluation professionals; senior academic 
administrators; department chairs or academic program administrators; student affairs 
professionals; and faculty advisors. 

4. Distribution of Reports: whether assessment reports are regularly distributed to students, 
faculty, academic administrators, student affairs professionals, employers, and the general public. 
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Figure 28 presents information on these four indices. Each was originally measured on a 
different scale. For the purposes of this graph, we have translated each index to a scale of zero to 
five. Therefore, we can determine which of these practices is more prevalent in baccalaureate and 
all institutions in our study. 

Figure 28. Extent of Institutional Use of Assessment Management Practices 




academic computer access to report 

budget support information distribution 



Scale: 0 = institution uses no practices in index; 5 = institution uses all practices in index. 



Of these four assessment management practices, baccalaureate institutions are most likely to 
provide access to individual students’ performance data to a number of internal constituents. For 
example, one of the items that comprises this index is whether student assessment data are 
accessible to senior academic administrators — 75% of baccalaureate respondents report that they 
are. If internal constituents have access to student assessment data, they should be able to make 
decisions based on these data, use the data to conduct studies, and even advise individual students 
based on their assessment information. 

The next most prevalent student assessment management practice is distributing student 
assessment reports to internal and external constituents. This index is comprised of whether 
assessment information is distributed to six individuals. Most baccalaureate institutions regularly 
distribute reports to academic administrators (91%), faculty (7 1 %), and student affairs 
professionals (51%). Fewer baccalaureate institutions regularly distribute reports to students 
(20%), the general public (7%), or employers (2%). 

Comparatively speaking, using computer information systems to manage the student 
assessment process is a less-often used practice. One of the items in this index is whether the 
institution has a student information system that tracks students from application through 
graduation. Less than half of baccalaureate institutions (40%) report that they have such a system. 
Even fewer have either a system that includes student performance indicators or is integrated with 
other institutional databases. 

Using the budget to compare. and reward units or personnel is practically non-existent. For 
example, only 3% of institutions report that they reward academic units for improvement based on 
student performance indicators. Compared to all institutions in our study, baccalaureate 
institutions are slightly less likely to use computer support for assessment processes and to provide 
access to assessment information. 
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Policies Supporting and Promoting the Use of Student Assessment 

While the previous section focused on practices used to manage student assessment, this 
section focuses on the policies used to both support student assessment and guide how the 
resulting data are incorporated into other institutional processes. We asked survey respondents 
about the extent to which they have developed a variety of specific institutional policies to support 
and promote student assessment in four areas: student involvement in assessment; professional 
development for student assessment; training for student affairs staff; faculty evaluation and 
rewards; and academic planning and review processes. Factor analysis produced five indices of 
student assessment policies, detailed information on these indices can be found in Appendix B 
Briefly, these five indices are: 

1. Student Involvement: extent to which the institution requires students to participate in student 
assessment activities; provides students with information on the purpose and uses of student 
assessment; and provides individual feedback regarding student performance results. 

2. Professional Development: extent to which the institution provides funds for faculty to attend 
assessment conferences; faculty workshops or consultative services on student assessment; 
assistance to faculty to improve use of student assessment; and workshops/seminars for academic 
administrators on assessment. 

3; Student Affairs Training: extent to which the institution requires student affairs staff to receive 
training on assessment and provides student assessment workshops for student affairs 
administrators. 

4. Faculty Evaluation: extent to which the institution considers evidence of student performance in 
faculty evaluation for promotion; incorporates evidence of student performance into faculty 
evaluation for salary and merit; considers faculty scholarship on assessment in promotion, tenure, 
or salary reviews; considers faculty participation in assessment in promotion, tenure, or salary 
reviews; and recognizes faculty for effective use of assessment. 

5. Academic Planning and Review: extent to which the institution incorporates student performance 
data into academic department or undergraduate program planning or review; general education or 
core curriculum review; course-level review and development; and review and planning for student 
academic support services. 

Figure 29 presents the means for these five indices for baccalaureate and all institutions. 
Scores for the individual items within each index are available in Appendix A. 

Of the five assessment management policies presented in Figure 29, baccalaureate 
institutions are most likely to incorporate student assessment data into academic planning and 
review processes (2.82). For example, respondents report that in many departments, student 
performance data are incorporated into undergraduate program planning or review and into general 
education or core curriculum review— two of the items comprising this index. This finding 
provides evidence that baccalaureate institutions are engaging in student assessment to improve 
undergraduate programs, a purpose they listed as very important. 

Baccalaureate institutions also make fairly extensive use of policies encouraging student 
involvement in assessment activities (2.79). For the three items that comprise this index, 
baccalaureate institutions report that many departments require students to participate in assessment 
activities and provide students with information while some departments provide students feedback 
on student assessment. Requiring participation and providing information on assessment purposes 
should increase student involvement, while providing individual feedback may improve student 
performance. 
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Figure 29* Extent of Institutional Use of Assessment Management Policies 
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Baccalaureate institutions are slightly more likely to offer professional development on 
student assessment to faculty and academic administrators (1.79) than they are to offer training and 
workshops for student affairs staff (1.66). Within the index of professional development, 
baccalaureate institutions report that some departments offer funds for faculty to attend assessment 
conferences. Only a few departments provide assistance to faculty in the form of paid leaves, 
stipends, and course reductions. 

Institutions are much less likely to evaluate and reward faculty based on student assessment 
participation or results (1.37). For example, respondents report that at their campus, less than a few 
departments consider faculty participation in assessment in promotion, tenure or salary reviews. 
Neither is student performance used to reward or evaluate faculty in more than a few departments. 
While these uses of student assessment data are fairly intrusive, one of the items in this index is more 
benign: publicly recognizing faculty for effective use of assessment. Nonetheless, again, only a few 
departments within baccalaureate institutions are using this strategy. 

In comparison with all institutions in our study, baccalaureate institutions are less likely to 
provide professional development on student assessment to academic administrators, faculty, and 
student affairs administrators. Perhaps funding for professional development initiatives is limited in 
baccalaureate institutions. Baccalaureate institutions are slightly more likely to involve students in 
student assessment, reward and evaluate faculty using student assessment information, and use 
student assessment information in program and course planning and review than are all institutions. 

Summary Observations 

Baccalaureate institutions make little use of student assessment information in making either 
budget or faculty evaluation and reward decisions. Less obtrusive policies, such as distributing 
assessment results and providing access to student performance information, are more prevalent. 
Compared to all institutions, baccalaureate institutions are less likely either to use computers in their 
assessment process or to provide access to student assessment information, findings that are likely 
related. Also, in comparison to all institutions, baccalaureate institutions are less likely to offer 
professional development to either faculty or to academic or student affairs administrators. 

However, they are more likely to involve students in the assessment process and to reward or 
evaluate faculty based on assessment participation or outcomes. 
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7. USES AND IMPACTS OF STUDENT ASSESSMENT " 

Explores two important questions regarding how institutions use student 
assessment information in decision making and the impact it has on 
institutions. Baccalaureate institutions are not using student assessment 
information to any great degree in making academic decisions and report that 
student assessment has had little impact on either their internal processes or 
their external relationships. 

As noted in our introduction, effective student assessment processes contribute to 
improvements in institutional and student performance. Consequently, two important questions in 
our research are: 1) how do institutions use student assessment information and 2) how does 
student assessment impact institutions? 

From the literature we identified and focused on two critical dimensions: the use of student 
assessment information in academic decision making, and the internal and external impacts on the 
institution that have resulted from student assessment. Our survey included ten academic decision 
and fourteen institutional impact items. Institutions’ scores for each use and impact item are 
presented in Appendix A and details on the indices are in Appendix B. We used factor analysis to 
create indices of student assessment uses and impacts in these five areas. Briefly, the five indices 
are: 

1. Educational Decisions: the extent to which student assessment, information is used in revising 
undergraduate academic mission or goals; designing or reorganizing academic programs or majors; 
designing or reorganizing student affairs units; allocating resources to academic units; modifying 
student assessment plans, policies, or processes; modifying general education curriculum; 
modifying student out-of-class learning experiences; creating or modifying distance learning 
initiatives; modifying teaching methods; and modifying student academic support services. 

2. Faculty Decisions: the extent to which student assessment information is used in decisions on 
faculty promotion and tenure; and salary increases or rewards. 

3. Faculty Impacts: whether student assessment has stimulated campus discussions of 
undergraduate education; contributed to faculty satisfaction; contributed to faculty interest in 
teaching; and led to changes in teaching methods used. 

4. Student Impacts: whether student assessment contributes to student satisfaction; affects student 
retention or graduation rates; affects student grade performance; and affects student achievement on 
external examinations. 

5. External Impacts: whether student assessment information affects student application or 
acceptance rates; allocation of state funding; evaluation from regional accreditation agency; private 
fund-raising results; success on grant applications; communications with external constituents; and 
institutional reputation or image. 

Figure 30 presents the means for these five indices for baccalaureate and all institutions. 

Uses in Academic Decisions 

Regarding academic decisions, baccalaureate institutions are slightly more likely to use 
student assessment information to make faculty related decisions (1.44) than to make educational 
decisions ( 1 .40). Within the faculty decision index, baccalaureate institutions are more likely to 
use student assessment information in decisions on promotion or tenure (1.70) than in decisions on 
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salary increases or rewards (1.49). Within the educational decision index, baccalaureate 
institutions are most likely to use student assessment information to design or reorganize academic 
programs or majors (2.61). Also within this index, baccalaureate institutions are least likely to 
create or modify distance learning initiatives ( 1 .52). 



Figure 30. Institutional Uses and Impacts of Student Assessment Information 




faculty educational student faculty external 

decision* decision* impact** impact** impact** 



*Use scale: l=no action or influence unknown, 2=action taken, data not influential; 3=action taken, data somewhat 
influential; 4=action taken, data very influential. 

Impact scale. l=not monitored, do not know; 2=monitored, negative impact; 3=monitored, no known impact; 
4=monitored, positive impact. 



Baccalaureate institutions are not making great use of student assessment information in 
making either faculty or educational decisions. However, compared to all institutions, they make 
greater use of student assessment information in faculty-related decisions and just as much use of 
student assessment information in educational decisions. Perhaps baccalaureate institutions are 
more likely to use student assessment information to make faculty decisions because of the high 
value they place on both teaching and quality in undergraduate education. 

Institutional Impacts 

In terms of the impacts of student assessment information, baccalaureate institutions are 
most likely to report that student assessment impacts faculty. Perhaps faculty are more likely to be 
impacted because these institutions are more likely to use assessment information in making faculty 
decisions related to promotions and salary increases. Within this index, institutions are most likely 
to report that student assessment has led to changes in teaching methods used (2.60), and least 
likely to report that student assessment has contributed to faculty satisfaction (1.88). Many student 
assessment techniques require greater faculty effort and involvement, which may not lead to greater 
faculty satisfaction, especially in the short-run. 

Baccalaureate institutions are less likely to report that student assessment has impacted 
students. Within this index, institutions are most likely to report that student assessment has 
affected student retention or graduation rates (2.26) and least likely to report that student 
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assessment has affected student grade performance (1.91). These findings are quite interesting; 
baccalaureate institutional respondents believe that student assessment results have led to students 
remaining enrolled at the institution longer. However, they are less likely to believe that students 
grades are higher due to assessment. Therefore, it appears that something other than better grades 
is contributing to student retention. Perhaps students become more involved with the institutions 
through assessment activities and are therefore more likely to remain enrolled longer. 

Institutions were least likely to say that student assessment had impacted their external 
relationships. The most commonly affected external domain is in the receipt of an evaluation from 
the regional accreditation agency (2.57). The least commonly affected external domain is the 
allocation of state funding (1.24). Most of the baccalaureate institutions that responded to this 
study are private. 

Overall, baccalaureate institutions report that student assessment has not greatly impacted 
either their internally- or their externally-oriented processes and functions. However, compared to 
other institutions, baccalaureate institutions are slightly more likely to report that their students and 
their external relationships are impacted by student assessment. They are considerably more likely 
than all institutions to report that their faculty are impacted by student assessment. 

Summary Observations 

Baccalaureate institutions are not using student assessment information to any great degree 
in making either faculty or educational decisions. However, they are more likely to use assessment 
information to make faculty decisions than are all institutions. Also, baccalaureate institutions do 
not report that student assessment has impacted either their internal processes or their external 
environment. However, compared to all institutions, they are more likely to report that their 
faculty members are impacted by student assessment. While it appears that student assessment 
information is not widely used by baccalaureate institutions, these institutions do seem to be doing 
more than other institutions are to involve faculty in assessment activities. 
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8. KEY RELATIONSHIPS 



Internal influences, such as a mission emphasis on student assessment and 
excellence in undergraduate education; conducting assessment for internal 
improvement purposes; and providing a number of administrative and 
governance activities to support student assessment have more influence on 
baccalaureate institutions’ involvement with student assessment than do 
external forces. Engaging in student assessment for both state and internal 
purposes; collecting extensive cognitive student data; using a wide variety of 
data collection methods; extensive faculty and administrative support; 
involving a great number of internal constituents through professional 
development; involving students in assessment; providing access to 
assessment information; and conducting studies to link students’ performance 
to their interactions with the institution are related to more extensive uses of 
student assessment data in making academic decisions. Baccalaureate 
institutions conducting evaluations of their assessment process; sponsoring 
extensive administrative and governance activities; offering professional 
development opportunities to student affairs staff; conducting studies linking 
student performance to their institutional interactions; using the student 
assessment data they collect to evaluate faculty and to plan and review 
academic programs; and collecting extensive cognitive data report that student 
assessment has had a greater impact on their internal processes and external 
relationships. 

In the previous sections we have examined the domains of external influences on, the 
institutional approaches to, support for, management policies and practices regarding, and uses and 
impacts of student assessment. Even more important is understanding how these domains 
influence positive uses and impacts of student assessment data within institutions. That is, which 
external influences, institutional approaches to assessment, organizational and administrative 
support patterns, and assessment management practices and policies are most likely to promote the 
use of and positive impacts from student assessment information? 

In this section, we will examine three key relationships. First, we discuss how external 
influences affect an institution’s approach to student assessment (the type and extent of their use of 
student assessment). Then we examine how external influences and institutional characteristics, 
approaches, support patterns, practices and policies affect an institution’s use of student 
assessment data. Finally we examine how these domains relate to positive institutional impacts 
from student assessment. (The information in this section uses the indices discussed in previous 
sections. For detailed information on the indices, please refer to Appendix B.) 

External and Internal Influences on Student Assessment Approaches 

Are external forces more influential than internal forces in determining how baccalaureate 
institutions approach student assessment? We used three multiple regression models to compare 
the influences of state characteristics related to student assessment, regional accreditation 
affiliation, and institution- wide support for student assessment on the extent of an institution’s use 
of three approaches to student assessment: cognitive assessment^ (tire extent to which data are 
collected on measures of students’ cognitive performance); affective assessment (the extent to 
which data are collected on measures of students’ affective development or performance); and post- 
college assessment (the extent to which data are collected on measures of students’ post-enrollment 
performance). In these three models, the predictor variables stem from three domains: state 
assessment approach, accrediting region, and institution-wide support. The predictor variables in 
this model are shown in Table 6 and defined in detail in Appendix B. 
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Table 6. The Influence of Institution-Wide Support, State Assessment Approach, and 
Accrediting Region on the Extent of Institutional Approach to Student Assessment for Public 
Baccalaureate Institutions 







Extent of Institutional Approach 




Cognitive 


Affective 


Post-College 




Assessment 


Assessment 


Assessment 




Beta 


AR 2 


Beta AR 2 


Beta AR 2 


Adjusted R 2 

Institution-Wide Support 


.17** 




.13** 


.05** 


Mission emphasis 






.16* .03 


.19** .04 


Administrative & governance activities 
Administrator & faculty support 


24** 


.09 


.23** .08 




Conduct for internal improvement 


22** 


.03 


.15* .02 




Conduct for state 
Conduct for accreditation 






-.16* .02 




State Assessment Approach 










Authority structure 

Form of state assessment initiative 

Common indicators/outcomes 








-.15* .02 


Accrediting Region 










Middle States 


-.16* 


.05 






North Central 
New England 
Northwest*** 
Southern 
Western 


.17* 


.02 


- 





*p < .05; **p < .01 

***Since “accrediting region” was a categorical variable. Northwest Accrediting Region was left out of this 
regression because its effect on cognitive competencies, based on ANOVA, was closest to the mean. 



Cognitive Assessment 

The first two columns in Table 6 show the regression results for the relationship of the 
predictor variables from the three domains on the extent of institutional use of cognitive 
assessment. This model is fairly strong, explaining 17% of the variance in the use of cognitive 
assessment. Which of the variables has the most influence on the extent to which baccalaureate 
institutions assess students’ cognitive abilities? The institution-wide patterns of support and the 
accrediting region are influential, while state characteristics are not. Within the institution-wide 
support domain, the extent of the institution’s administrative and governance activities and an 
institutional emphasis on student assessment for internal improvement purposes are both important 
predictors of the extent of assessing students’ cognitive abilities. Offering administrative and 
governance activities and assessing students for internal purposes both belie a dedication to the 
assessment process. This dedication evidences itself in the greater extent of cognitive data 
collected. In terms of accrediting regions, institutions in the North Central regional accrediting 
region are more likely to assess students’ cognitive abilities, while institutions in the Middle States 
region are less likely to do so. 



Affective Assessment 

The middle columns in Table 6 show the regression results for the relationship of the 
predictor variables in the three domains to the extent of institutional use of affective assessment. 
This model is not as strong, explaining only 13% of the variance in the collection of affective data. 
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The extent of affective assessment conducted by baccalaureate institutions is solely affected by 
institution-wide support patterns. The extent of administrative and governance activities offered, a 
mission emphasizing assessment, and conducting student assessment for internal purposes are ail 
important influences on affective assessment. These three actions demonstrate that the institution is 
dedicated to the student assessment process; this dedication evidences itself in the greater collection 
of affective data. Interestingly, if an institution reports that state mandates are an important reason 
for conducting student assessment, it will collect less affective data. Perhaps states are less likely 
to be asking institutions to collect affective data. 

Post-College Assessment 

Finally, the last two columns in Table 6 present the regression results for the relationship of 
the predictor variables in the three domains on the extent of institutional use of post-college 
assessment. This model is the weakest of the three, explaining only 5 % of the variance in the 
collection of post-college data. There are only two significant predictors in this model. First, 
having an academic mission that emphasizes assessment and excellence in undergraduate education 
influences the amount of post-college data collected. Institutions that prioritize assessment and 
undergraduate education are likely to have extensive data collection efforts that include a focus on 
former students. Second, in states that are active in setting policies and writing legislation on 
student assessment, baccalaureate institutions are less likely to collect post-college data. Perhaps 
states are not as concerned with post-college information as they are with information on currently 
enrolled students at baccalaureate institutions. 

Influences on Using Assessment Information in Academic Decisions 

How do baccalaureate institutions effectively promote and support the use of student 
assessment information in academic decision making? Within the domain of academic decision 
making, we created two indices reflecting the use of student assessment information: educational 
decision making and faculty decision making (these indices are described in detail in Appendix B). 
We used multiple regression to examine the influences of external forces, institutional size, 
institutional approach to student assessment, institution-wide support, and assessment management 
practices and policies on using student assessment information in both educational and faculty 
decisions. The predictor variables in this model are defined in detail in Appendix B. Table 7 
presents the results of these two regression models for baccalaureate institutions. 

Influence on Use of Assessment Information in Educational Decisions 

The educational decisions model works well for baccalaureate institutions, explaining 40% 
of the variance in the influence of student assessment data on educational decisions. While there 
are significant predictor variables from the domains of assessment approach, institution-wide 
support, and practices and policies, most of the significant variables are from the assessment 
management practices and policies domain. 

Within the domain of institution-wide support for student assessment, baccalaureate 
institutions that conduct student assessment for internal purposes are more likely to use the 
resulting student assessment data in making educational decisions. Within the domain of 
assessment management practices and policies, institutions that use assessment information to 
evaluate and reward faculty, involve students in the assessment process, and offer professional 
development on student assessment to faculty, academic administrators and student affairs staff 
and administrators are more likely to use student assessment information to make educational 
decisions. Within the assessment approach domain, the more studies baccalaureate institutions 
conduct on the link between students’ performance and their educational experiences, and the more 
instruments they use in collecting assessment data, the more likely they are to use student 
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Table 7. The Influence of External Influences, Institutional Context, Institutional Approach, 
Institution-Wide Support, and Management Practices and Policies on Using Student 



Assessment Information in Educational and 


Faculty 


Decisions for 


Baccalaureate 


Institutions 




Educational Decisions 


Faculty Decisions 




Beta 


AR 2 


Beta 


AR 2 


Adjusted R 2 




.40** 


.31** 


External Influences 










Middle States accrediting region 
North Central accrediting region 
New England accrediting region 
Southern accrediting region 
Western accrediting region 
State initiative for student assessment 






-.14** 


.02 


State approach to student assessment 
Accreditation influence 










Institutional Context 










Enrollment 

Control (1 = public, 2 = private) 






.12* 


.01 


Institutional ADDroach to Student Assessment 










Cognitive assessment 






.17** 


.03 


Affective assessment 








. . . 


Post-college assessment 
Number of instruments 
Student-centered methods 


.13** 


.02 


.17** 


.02 


External methods 
Total assessment studies 


.16** 


.03 


.17** 


.06 


Institution-Wide Support for Student Assessment 










Mission emphasis 

Conduct for internal improvement 

Conduct for accreditation 


.19** 


.09 


-.17** 


.04 


Conduct for state 






.21** 


.02 


Administrative & governance activities 
Administrator & faculty support 






.13** 


.02 


Formal centralized policy 
Institution-wide planning group 
Conducted evaluation of assessment process 






-.10* 


.01 


Assessment Management Practices and Policies 










Academic Budget decisions 
Computer support 
Access to information 
Distribution of reports 
Student involvement 


.16** 


.03 


.14** 


.09 


Professional development 


.12* 


.17 






Student affairs training 


.12* 


.01 






Faculty evaluation 1 


J8** 


.05 


n/inc 




Academic planning & review 2 


n/inc 









*p < .05; **/? < .01 _ 

'The factor “faculty evaluation” was not entered into the regression model predicting use of student assessment 
information in making faculty decisions, since many of the items comprising these two factors were similar. 

^The factor “academic planning and review” was not entered into the regression model predicting use of student 
assessment information in making educational decisions, since many of the items comprising these two factors were 
similar. 
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assessment information to make educational decisions. Most of these significant predictor 
variables symbolize a commitment to the assessment process. Therefore one would expect that 
those institutions that are committed to the assessment process would use the resulting information 
to make educational decisions. Professional development is an important predictor variable, as the 
more internal constituents know about student assessment, the more likely they will be to use the 
resulting information in making educational decisions. Similarly, the number of studies conducted 
by an institution is an important predictor variable, as the more studies the institution does on the 
link between students performance and their interactions with the institution, the more direction 
internal constituents will have in making decisions about academic policies and practices. 

Influences on the Use of Assessment Information on Faculty Decisions 

The faculty decisions model also works well for baccalaureate institutions, explaining 3 1 % 
of the variance in making faculty-related decisions. In our regression model, there are significant 
predictor variables from all five domains. The domain of institution-wide support for student 
assessment has the most significant predictor variables. Baccalaureate institutions that report that 
meeting state mandates is an important purpose for conducting assessment are more likely to use 
assessment information in faculty decisions. Conversely, baccalaureate institutions that report that 
meeting accreditation mandates is an important purpose for conducting assessment are less likely to 
use assessment information to make faculty decisions. Baccalaureate institutions that have strong 
faculty and administrative support for student assessment are more likely to use assessment 
information in making faculty decisions. Those institutions that have conducted an evaluation of 
their assessment process are less likely to be using assessment information in faculty decisions. 

There are three significant variables from the institutional approach domain. The greater the 
use of both external and student-centered methods to collect assessment data, the more likely the 
institution is to use student assessment data to make faculty-related decisions. In addition, the 
greater the extent of cognitive data collected, the more likely the institution is to use student 
assessment data in making faculty decisions. 

There is one significant predictor variable from the assessment management practices and 
policies domain. Institutions that provide broad access to student assessment information are more 
likely to use student assessment data to make faculty-related decisions. In the external influences 
domain, institutions in the North Central accrediting region are less likely to use student 
assessment information in making faculty-related decisions. Finally, in the institutional context 
domain, private baccalaureate institutions are more likely to use student assessment information to 
make faculty-related decisions. 

Interestingly, state mandates appear to spur institutions to make faculty-related decisions, 
while accreditation mandates have the opposite effect. Perhaps states are more concerned with 
holding faculty accountable for student learning at baccalaureate institutions than are accrediting 
agencies. The more data an institution has gathered, the more likely they are to use these data to 
make faculty-related decisions. One would expect that a proliferation of data is necessary for such 
decisions. Administrative and faculty support are also important predictors of using assessment 
data to make faculty-related decisions. This finding is expected, as using assessment data to make 
faculty-related decisions can be controversial. 

Influences of Student Assessment Information on Internal and External 

Institutional Performance" 

How does the use of student assessment information affect various internal and external 
institutional performance dimensions? In our survey, baccalaureate institutions reported whether, 
and the extent to which, student assessment has impacted various aspects of faculty and student 
performance and their relationships with the external environment. We created three indices 
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reflecting the impact of student assessment information on faculty, students, and the institutions’ 
external relations (these indices are described in detail in Appendix B and in section seven). We 
used multiple regression to examine the influences of external forces, institutional size, institutional 
approach to student assessment, institution-wide support, and assessment management practices 
and policies on the impact of student assessment information on students, faculty, and the 
institution’s external environment. The predictor variables in this model are defined in detail in 
Appendix B. Table 8 presents the results of these three regression models for baccalaureate 
institutions. 

Influences on the Impact of Student Assessment Information on Faculty 

What variables predict how extensive an impactstudent assessment will have on faculty and 
their activities? The first two columns of Table 8 show the statistically significant predictors of 
faculty impacts. The model on faculty impacts is fairly strong, predicting 29% of the variance in 
the impact student assessment information has on faculty. Institution-wide support for student 
assessment is key, according to the results of our regression model. If baccalaureate institutions 
offer administrative and governance activities on student assessment, evaluate their assessment 
process, and place an emphasis on student assessment in their mission statement, it is more likely 
that their assessment activities will impact faculty. 

The assessment management practices and policies used by the institution are also 
important determinants of whether their assessment activities will affect faculty. Institutions that 
use student assessment information in their academic planning and review process are more likely 
to report that their assessment activities have impacted faculty. However, institutions that use 
student assessment information to make budget allocation decisions are not likely to report that 
student assessment information has had a positive impact on faculty. 

Within the institutional approach domain, baccalaureate institutions that conduct studies 
linking students’ performance to their interactions with the institution are likely to report that their 
assessment activities have impacted faculty. No variables from either the external influences or the 
institutional context domains were significant in this model. 

Administrative and governance activities, evaluations of student assessment processes, and 
academic planning and review processes are all activities that are likely to involve faculty. It seems 
reasonable that faculty involvement in these activities would lead to a greater impact on faculty by 
student assessment information. It is also somewhat expected that faculty would not be positively 
impacted by assessment information in institutions that use assessment data in making budget 
decisions. These actions are likely to be controversial. Finally, it may be expected that institutions 
that conduct studies linking students’ performance to their interactions with the institution would 
report a positive impact on faculty by their student assessment information. The more data faculty 
have to work with, the more likely they are to be impacted by these data. 

Influences on the Impact of Assessment Information on Students 

The middle two columns on Table 8 show which variables are statistically significant 
presictors of achieving student impacts from assessment in baccalaureate institutions .The model 
on student impacts is slightly weaker than the model on faculty impacts, explaining 24% of the 
variance in the impact of assessment data on students. Within the assessment management 
practices and policies domain, institutions that use student assessment data in their academic 
planning and review process and those that train student affairs staff in assessment are likely to 
report that their assessment activities have impacted students. Within the institution-wide support 
domain, institutions that conduct evaluations of their assessment process are likely to report that 
their assessment information has impacted students. However, those institutions that have a 
formal centralized policy for assessment are not likely to report that their assessment information 
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Table 8. The Influence of External Influences, Institutional Context, Institutional Approach, 
Institution-Wide Support, and Management Practices and Policies on Faculty, Student, and 
External Impacts for Baccalaureate Institutions 





Faculty Impacts 


Student Impacts 


External Impacts 




Beta 


AR 2 


Beta 


AR 2 


Beta 


AR 2 


Adjusted R 2 

External Influences 




.29** 


• 


24** 


• 


22** 


Middle States accrediting region 
North Central accrediting region 
New England accrediting region 
Southern accrediting region 
Western accrediting region 
State initiative for student assessment 
State approach to student assessment 
Accreditation influence 










-.12* 


.02 


Institutional Context 












Enrollment 














Control ( 1 =public, 2-private) 

Institutional Approach to Student Assessment 














Cognitive assessment 
Affective assessment 
Post-college assessment 
Number of instruments 
Student-centered methods 
External methods 
Total assessment studies 


.24** 


.14 


.19** 


.05 


.17** 


.02 


Institution-Wide Support for Student 
Assessment 














Mission emphasis 

Conduct for internal improvement 

Conduct for accreditation 


.10* 


.01 






.11* 


.01 


Conduct for state 














Administrative & governance activities 
Administrator & faculty support 
Formal centralized policy 
Institution-wide planning group 


.21** 


.04 


-.12* 


.01 


-.12* 


.01 


Conducted evaluation of assessment process 
Assessment Management Practices and Policies 


.11* 


.01 


.13* 


.01 


.16** 


.03 


Academic budget decisions 
Computer support 
Access to information 


-.11* 


.01 










Distribution of reports 
Student involvement 














Professional development 
Student affairs training 
Faculty evaluation 






.18** 


.03 


.14* 


.03 


Academic planning & review 

*r» ^ fK. ^ n 1 


.22** 


.07 


.24** 


.12 


.16** 


.10 



*p < .05; **p < .01 
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has impacted students. Within the approach domain, the more studies baccalaureate institutions 
conduct on linking students’ performance to their interactions with the institution, the more likely 
student assessment is to impact students. 

It would be expected that conducting studies linking students’ performance to their 
institutional interactions and using student assessment data to plan and review academic courses 
and programs would eventually impact students. It is interesting that institutions that train their 
student affairs staff in assessment report a greater impact by their assessment data on students. . 
Perhaps student affairs staff have more contact with students and can explain assessment benefits 
to them. It is also interesting that in institutions with formal centralized policies for assessment, 
students are less likely to be impacted by assessment information. It is likely that in these 
institutions, individual students do not receive feedback on their assessments as the processes are 
centrally controlled and thus further removed from the faculty and administrators who have contact 
with students. 

Influences on the Impact of Assessment Information on External Institutional Relations 

What variables predict how extensive an impact student assessment will have on a 
baccalaureate institution’s external relations? The last two columns in Table 8 demonstrate that the 
model on external impacts is slightly less powerful than the model on student impacts, explaining 
22% of the variance in the external impact from student assessment information. Within the 
approach domain, the more cognitive data collected by the institution, the more likely these data are 
to have an external impact. Within the assessment management practices and policies domain, 
institutions that use student assessment information in their academic planning and review 
processes and in evaluating faculty are more likely to report that engaging in student assessment 
has had an external impact. Within the domain of institution-wide support, institutions that have 
conducted an evaluation of their assessment process and those that emphasize assessment in their 
mission statement are more likely to report that engaging in student assessment has impacted their 
external environment. Those that have an institution-wide planning group for assessment are less 
likely to report that their assessment information has had an external impact. Within the external 
influences domain, institutions that say their accrediting agency has influenced their assessment 
activities are less apt to report that their assessment information has had an external impact. 

It is interesting that institutions that use student assessment information in evaluating 
faculty are more likely to report that engaging in student assessment has had an external impact. 
Perhaps state mandates require faculty evaluations based on student performance and those 
institutions that comply report favorable impacts of such compliance. It also makes sense that 
those institutions that have evaluated their assessment process report that their assessment 
information has positively impacted their external relationships. If institutions can demonstrate the 
effects of their assessment program, it is highly likely that they will use these findings to impress 
and to improve their relationships with external evaluators. 

Summary Observations 

In examining how internal and external forces compare in influencing baccalaureate 
institutions’ approaches to assessing students, it is clear that internal influences and accrediting 
region are more influential than are state characteristics. In instances where state characteristics are 
significant, they tent to have a negative effect on the extent of data collection at institutions. Three 
internal strategies are particularly important in determining the extent of student data collected: 
emphasizing student assessment and excellence in undergraduate education in the mission 
statement, conducting assessment for internal improvement purposes, and providing a number of 
administrative and governance activities on student assessment. 
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If baccalaureate institutions are to benefit from engaging in student assessment, they should 
not only be collecting student data, but also using these data to make academic decisions. 
Educationally-oriented decisions were influenced by a different set of variables than were faculty- 
related decisions. For educational decisions, there were five important predictors in our model: 
assessing students for internal improvement purposes; using assessment information to evaluate 
and reward faculty; involving students in the assessment process; offering professional 
development for faculty, academic administrators, and student affairs staff; and conducting studies 
on the link between students performance and their interactions with their institutions. Institutions 
that intend to use assessment data for improvement purposes and that conduct studies to better 
understand how their actions influence student performance are in a better position to use student 
assessment data in making educational decisions. 

For faculty-related decisions, there were six important predictors in the model: assessing 
students to meet state requirements; having faculty and administrative support for assessment; 
evaluating their assessment process; using both external and student-centered methods; collecting 
cognitive data; and providing broad access to data. If bacvcalaureate institutions are assessing 
students to meet state requirements, states may be requiring that assessment data be part of faculty- 
related decisions. Furthermore, institutions that want to use student assessment data in making 
faculty decisions likely would need to have strong faculty support for the assessment process. In 
addition to these six variables, private institutions are more likely to use assessment data in making 
faculty decisions. Conversely, those institutions that report that they conduct assessment to meet 
accreditation requirements are less likely to use assessment information in making faculty 
decisions. 

If engaging in student assessment makes a difference in baccalaureate institutions, they 
should report that it is impacting their internal processes and external relations. Within institutions 
reporting impacts from their assessment process^extemal influences were not important predictors 
with one exception — if the institution conducted assessment to meet accreditation requirements, 
they were not likely to report that their assessment activities had an external impact. In these cases, 
it is likely that the accreditation agency is influencing the institution more than the institutions’ 
assessment activities are influencing their accreditation process. The existence of an assessment 
planning group also has a negative influence on external impacts. Perhaps these groups focusing 
their efforts on internal rather than external uses of assessment information. Positive influences on 
external impacts include the extent of cognitive assessment data collected and whether the data has 
been used to evaluate and reward faculty. Perhaps institutions that demonstrate that they use 
assessment data to evaluate faculty are strengthening relationships with external constituents such 
as state representatives. 

There were five important variables that predicted whether student assessment information 
impacted faculty at baccalaureate institutions: the extent of administrative and governance activities 
offered; whether the assessment process had been evaluated; whether the mission statement 
emphasized assessment; whether assessment information was used in planning and reviewing 
academic programs and courses; and whether the institution had conducted studies linking student 
l^rformance to institutional activities. It may be expected that institutions that offer regular 
activities on assessment would have greater faculty involvement. Furthermore, if assessment 
information is used to plan and review programs and courses, faculty will undoubtedly be affected 
by this assessment information. Finally, if, through their studies of students, institutions find 
links between student performance and faculty-student interactions — or some other faculty-oriented 
activity — faculty will likely be affected by these findings. 

According to our results, students are more likely to be impacted by the assessment process 
if student affairs administrators and staff are trained in assessment. Perhaps students have more 
contact with these staff members. In addition, students are less likely to be affected by the 
assessment process if there is a centralized formal policy on student assessment at the institution. 
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It may be that in institutions with these policies, assessment data are both collected and distributed 
on an institution-wide basis, rather than allowing smaller individual units to collect and distribute 
their own data. If smaller units are in charge of assessing their own students, the students 
themselves may be more likely to receive feedback on their own assessments. Similarly, faculty in 
these smaller units may make changes to their curriculum based on assessment results which 
would also affect students. 
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9. EXECUTIVE SUMMARY: RESEARCH RESULTS AND RECOMMENDATIONS 

Recommendations for baccalaureate institutions based on the results of our 
research are outlined in this section. This study and monograph highlight the 
importance of viewing student assessment from a critical institutional 
perspective and underscore three important realities: 1) for student 

assessment to enhance student, faculty, academic, and institutional 
performance, it has to be viewed as an institutional process, not just a series 
of student assessment activities; 2) a great deal of organizational, 
administrative, and academic activity is or can be invested in initiating, 
managing, and using student assessment; and 3) that a systematic look at 
those institutional activities can enhance an institution ’s ability to plan for and 
use student assessment effectively. 

Student Assessment: A Critical Institutional Perspective 

The intent of our survey and of this monograph is to highlight the importance of viewing 
student assessment from a critical institutional perspective and to underscore three important 
realities: 1) for student assessment to enhance student, faculty, academic, and institutional 
performance, it has to be viewed as more than just a series of student assessment activities; 2) a 
great deal of organizational, administrative, and academic activity is or can be invested in initiating, 
managing, and using student assessment; and 3) a systematic look at those institutional activities 
can enhance an institution’s ability to use student assessment effectively. 

The data reported in this survey provide a national profile of what baccalaureate institutions 
are currently doing to support and promote the use of student assessment. This evidence provides 
insight into the activities institutions are doing extensively, those which merit greater attention, and 
those which make a difference in improving institutional performance. We begin with a summary 
of our research findings and recommendations concerning specific assessment activities conducted 
within baccalaureate institutions. Then we discuss how baccalaureate institutions can use the 
Institutional Support for Student Assessment (ISSA) inventory and the Framework for Institutional 
Support for Student Assessment (Figure 1 of this monograph) as a guide for examining their 
student assessment process and functions. We conclude by suggesting that the results of the 
national survey used in conjunction with institutional self-examination can serve as the basis for 
redesigning or planning a student assessment process which can enhance an institution’s academic 
performance. 

Student Assessment in Baccalaureate Institutions: Results and Recommendations 

from the National Survey 

Institutions that have committed resources to assessing the development and performance 
of their undergraduate student assessment activities should be able to use the student assessment 
data they collect in organizational decision making and to document impacts from their assessment 
activities. Baccalaureate institutions that are engaging in student assessment, yet do not feel that 
they are benefiting from the process as much as they could be, may want to adopt the strategies of 
institutions that have reported using and being positively impacted by assessment information. Our 
research has demonstrated that most baccalaureate institutions haye made only limited use of 
student assessment data and given little attention to monitoring their impacts. However, they have 
adopted a wide variety of student assessment approach measures, institution-wide activities 
supporting student assessment, and assessment management practices and policies. Our research 
identified strategies from the domains of student assessment approach, institution-wide support for 
student assessment, and assessment management practices and policies that are associated with 
baccalaureate institutions reaping greater institutional uses and impacts from undergraduate student 
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assessment. In the following sections, we highlight the student assessment activities that 
baccalaureate institutions are currently engaging in to a great extent, the activities "that they may 
want to augment, and the activities that are critical to enhancing the use of student assessment data 
in academic decisions leading to positive institutional impacts. 

Student Assessment Uses and Impacts 

If institutions are benefiting from their student assessment process, they should report that 
they are using resulting assessment data in academic decisions and that the use of these data has a 
positive impact on both internal processes and external relationships. In this section, we describe 
the extent to which baccalaureate institutions are using student assessment data and the extent to 
which these data have had an impact on the institution. Suggestions for increasing institutional 
uses and impacts of student assessment data are outlined in the following sections. 

Uses of Student Assessment Information . Most baccalaureate institutions are not using 
student assessment data to make academic decisions. Our study did not attempt to discern why 
institutions are not making more extensive use of these data. Perhaps institutional decision-makers 
do not have sufficient data to inform decisions concerning institutional policies and practices. If 
data collection focuses on student inputs (e.g., basic college-readiness skills) or outputs (e.g., 
graduation rates), without attempting to connect these inputs and outputs to institutional causes, 
decision makers may not be able to use this information to make changes. Assessment data use 
may also be limited if institutions have not created a formal mechanism for incorporating this data 
in decision making processes. Baccalaureate institutions should examine why they are not making 
more use of student assessment data. For example, is the type of data being collected not useful 
for informing decisions? Are there concerns about data quality? Is assessment data not easily 
accessible or not widely distributed to decision makers? Raising questions of this nature will assist 
institutions in understanding how they may increase the use of assessment information in decision 
making. 

Impacts of Student Assessment Information . Similarly, baccalaureate institutions are 
reporting very minimal impacts from student assessment data. For the most part, we found that 
baccalaureate institutions are simply not monitoring whether student assessment information has 
had an impact on either their students, their faculty, or their relationships with external bodies. 

Such monitoring is important if institutions are to evaluate the institutional benefits of engaging in 
student assessment. Suggestions for increasing institutional uses and impacts of student 
assessment data are outlined in the following sections. 

External Influences 

External bodies, such as state legislators and accreditation associations, influence 
institutions to engage in student assessment. In addition, a variety of postsecondary organizations 
and external funding sources offer support for institutions’ assessment programs. This section 
describes such influences on baccalaureate institutions and provides suggestions for deriving the 
most benefit from these external relationships. 

State Influences . Most of the baccalaureate institutions that responded to our survey are 
private institutions and are not under the purview of a state mandate for student assessment. 
However, for those baccalaureate institutions that are subject to a state mandate, most have been 
involved in developing the mandate. However, some institutions were not involved in this 
development. Where such opportunities exist, institutions should try to be involved with such 
planning at the state level in order to craft mandates that are fair, practical, and beneficial to the 
institutions. According to our research, institutions located in states with strong assessment 
mandates emanating from both a policy and a legislative source are less likely to collect extensive 
student assessment data on former students. Institutions in these states may want to examine their 
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approaches to student assessment and determine whether they need to augment data collection 
activities for internal improvement purposes regardless of the state mandate requirements. Our 
research also found that baccalaureate institutions that report that they conduct assessment to meet 
state mandates are more likely to use their assessment data in making faculty-related decisions. 
Perhaps these states are requiring a link between assessment outcomes and faculty promotion. 

Accreditation Influences . Although most baccalaureate institutions are not subject to a state 
mandate for student assessment, most have completed a regional accreditation review requiring 
undergraduate student assessment. For most baccalaureate institutions, accreditation requirements 
were either an important reason to initiate student assessment or an incentive to increase 
involvement in assessing students. Accreditation agencies apparently have a strong influence on 
baccalaureate institutions’ student assessment activities. Some regional accreditation associations 
have more influence on institutions’ student assessment activities than do others. For example, 
institutions in the Middle States accrediting region collect less student assessment data than do 
institutions in the North Central accrediting region. However, institutions in the North Central 
accrediting region are less likely to make use of student assessment data. To some extent, these 
differences reflect differences in the approach regional accreditors have taken toward student 
assessment. Institutions should be cognizant of the influences of their accrediting regions. While 
it is important to respond to accreditation requirements, baccalaureate institutions should also keep 
in mind that our research shows that if responding to accreditation requirements is the only major 
purpose for engaging in student assessment activities, these activities will not have an impact on 
the institution. That is, information collected and reported should be used not just to satisfy 
external reports and requirements but also for internal institutional improvement. 

Other External Influences . In addition to state mandates and accreditation requirements, 
there are other externally-oriented considerations bearing on baccalaureate institutions’ student 
assessment activities. Many baccalaureate institutions have made use of conferences and 
publications on student assessment provided by professional associations and regional 
accreditation associations. However, compared to all institutions in our study, baccalaureate 
institutions are less likely to receive state grants to help subsidize student assessment activities. 
While this difference is likely due to the predominance of private baccalaureate institutions in the 
study, those institutions that are under a state mandate for assessment could be more proactive in 
seeking state funding to help subsidize their assessment initiatives. 

Institutional Approach to Student Assessment 

Institutional approach to student assessment refers to institutional decisions regarding the 
collection and analysis of student assessment information. Dimensions along which assessment 
approaches can be differentiated include: the type and extent of student assessment data collected; 
the methods used to collect assessment data; and the analyses conducted and reported for collected 
data. Baccalaureate institutions have adopted a variety of approaches to collecting and analyzing 
student assessment data. This section describes these approaches and makes suggestions for 
adopting assessment approaches that will lead to institutional uses and impacts of student 
assessment information. 

Type and Extent . Baccalaureate institutions are collecting student assessment data from 
many students on their basic college-readiness skills, their satisfaction with their institutional 
experiences, their academic intentions, and their academic progress. While collecting these data is 
important, they reflect a concern for student academic placement, student satisfaction, and 
movement through the educational process, rather than cognitive or affective learning or 
development. In order for baccalaureate institutions to understand how their students are learning 
and changing while in college, they should broaden their data collection efforts. Currently, 
baccalaureate institutions only collect a limited amount of data on higher order skills, vocational 
skills, and the civic and social roles taken on by former students. Collecting data on higher-order 
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skills, such as critical thinking and problem solving, would allow baccalaureate institutions to 
move beyond the accountability-oriented data they currently collect (e.g., how many students are 
satisfied, how many graduate, and how many go on to further their education) to the more 
substantive data of cognitive growth. Not only will collecting such data provide a richer 
understanding of student growth, but our research found that the more cognitive data baccalaureate 
institutions collect, the more likely they are to use assessment data in making faculty-related 
decisions and the more likely it is that assessment data will have an impact on institutions’ external 
relationships. 

Assessm ent Methods . When collecting student assessment data, baccalaureate institutions 
as a whole tend to use tests and other written instruments. However, some units within 
baccalaureate institutions appear to be more innovative in terms of the methods they use to collect 
student assessment data. Respondents report that some units use observations of student 
performance; student portfolios or comprehensive projects; student performance in capstone 
courses; student interviews or focus groups; employer interviews or focus groups; alumni 
interviews or focus groups; surveys or interview with withdrawing students; transcript analysis; 
and external examination of students. Baccalaureate institutions should increase the use of these 
alternative measures of collecting student assessment data in order to gather data that is not 
accessible via more traditional tests and surveys. Such alternative measures also tend to promote 
faculty involvement in assessment through participation in designing and administering these 
instruments, and interpreting the data collected through them. In addition to gaining a better 
understanding of student growth and strengthening faculty involvement, using a number of 
assessment methods will benefit the institution. Our research found that the more extensive the 
data collection methods an institution employs, in terms of using a greater number of instruments 
and tests and using more student-centered and external methods, the more likely institutions are to 
use assessment data in academic decision making. 

Assessment Studies . Institutions in our study also reported whether they attempted to 
examine what aspects of the institution affect students’ performance. Two-thirds of baccalaureate 
institutions do conduct such studies. The most frequently conducted studies examine admissions 
standards/policies and extra-curricular activities. The least frequently conducted studies are on 
classroom, library, and/or computing resources; exposure to different instructional or teaching 
methods; and patterns of student-faculty interaction. These latter types of studies are important as 
they attempt to discern faculty and teaching impacts on students’ performance. Our research has 
found that institutions that conduct more studies report that their student assessment information 
has been useful in making educational decisions and that this information has had an impact on 
both faculty and students. Baccalaureate institutions should increase the number of studies they 
conduct. 

Institution-Wide Assessment Support Strategy 

Baccalaureate institutions have used a variety of organizational and administrative strategies 
to support their student assessment efforts, such as including student assessment within the 
academic mission; conducting assessment for internal improvement purposes; sponsoring 
institution-wide activities to promote involvement in and support for assessment; adopting an 
institutional plan or policy for student assessment; establishing processes and structures for 
planning and coordinating assessment; and evaluating the assessment process. This section 
summarizes such approaches adopted by baccalaureate institutions and offers recommendations 
regarding specific support strategies that enhance the likelihood of achieving institutional benefits 
from student assessment information. 

Mission Emphasis . One way to determine the degree of institutional support for an activity 
is to examine whether the activity is emphasized in the mission statement. While most 
baccalaureate institutions emphasize excellence in undergraduate education and describe intended 
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outcomes for student assessment, less than one-fifth accentuate student assessment in their mission 
statements. If student assessment is indeed a core value of the institution, this value should be 
communicated via the mission statement. Our research found that institutions that emphasize 
assessment in their mission statements report that their student assessment information has had a 
positive impact on both faculty and their external relationships. Thus, leaders in these institutions 
should consider reviewing the content and emphasis of their academic mission and including 
explicit mention of student assessment if that does not currently exist. 

Assessment Purposes . Similarly, support for assessment can be detected by understanding 
an institution’s purpose for engaging in student assessment. Baccalaureate institutions report that 
accreditation mandates and improving both undergraduate programs and student performance are 
all very important purposes for engaging in assessment. This finding is encouraging as it indicates 
that institutions both are aware of external demands and are purposefully engaging in student 
assessment to improve internal processes. Baccalaureate institutions should continue to maintain 
such a balance, especially as our research found that institutions that conduct assessment for 
internal improvement purposes report that they collect more cognitive and affective data and that 
this assessment data is used in making educational decisions. 

Institution-Wide Activities . Another measure of support for student assessment is the 
number of institution-wide administrative and governance activities used to promote assessment. 
Most baccalaureate respondents report that they have a faculty governance committee that regularly 
addresses assessment issues. Half of the respondents report that they provide regular workshops 
for academic and student affairs administrators. However, these institutions seldom provide 
incentives or rewards to administrators for engaging in or using the results of student assessment. 
Baccalaureate institutions could increase the use of internal accountability-oriented activities if they 
want to demonstrate institution-wide support for assessing students. Not only will offering such 
activities demonstrate support, but our research found that offering a plethora of administrative and 
governance activities is a positive predictor of collecting more cognitive and affective data and of 
these data having a positive impact on faculty members. 

Administrative and Faculty Support . Our survey also asked the respondent to describe his 
or her opinion of the level of support for student assessment given by various internal constituents. 
Only academic affairs administrators were described as very supportive. Does this finding indicate 
a feeling of ambivalence toward assessing students? Even the chief executive officer was 
described, on average, as being only somewhat supportive of student assessment. If the chief 
executive officer is not very supportive of student assessment, there are bound to be difficulties in 
promoting it, funding it, and rewarding people for engaging in it. In addition, faculty governance 
was described as only somewhat supportive. This finding is also cause for concern. Our research 
found that the more supportive faculty and administrators are of student assessment, the more 
likely it is that the institution will use student assessment data to make faculty-related decisions. 
Periodically measuring degrees of internal support for student assessment may be a useful means 
for baccalaureate institutions to gauge the effectiveness of efforts to promote assessment. 

Institutional Plans and Policies for Assessment . The types of plans and policies institutions 
develop for assessing students are telling of their support for student assessment. Virtually all 
responding baccalaureate institutions have some type of plan or policy for assessing students. The 
fact that they have plans and policies is encouraging as it indicates that they have devoted time and 
attention to the student assessment process. Half of baccalaureate institutions have developed a 
formal centralized plan that specifies assessment activities for all academic programs. The next 
most popular form is a formal decentralized policy that requires academic units or programs to 
develop their own assessment process. This form is followed in popularity by having a guidance 
policy that identifies institution-wide activities to be conducted by a central office or committee. 
Apparently baccalaureate institutions favor having some type of formal plan or policy regardless of 
whether it is centralized or decentralized. However, our research found that those institutions with 
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centralized policies report that their assessment results have less of an impact on students. 
Therefore, institutions may want to consider adopting decentralized policies in order to increase the 
likelihood that their assessment activities will impact students. 

Planning and Co ordinating Assessment . Most baccalaureate institutions have some type of 
planning group for student assessment. The majority of these groups are staffed by academic 
administrators and faculty members, with over half of the groups including an institutional 
researcher. Approximately one-third of these planning groups has a student representative. 
Baccalaureate institutions may want to consider increasing student involvement in these groups in 
order to understand how students feel about assessment. Once institutions understand how 
students view assessment, they can attempt to overcome student objections to participating in 
assessment activities. Most of the assessment planning groups are chaired by an academic 
administrator and approximately one-third of them are chaired by a faculty member. Similarly, the 
chief academic officer most often has approval authority for changing student assessment plans or 
policies, and an academic senate has such approval in half of the baccalaureate institutions 
surveyed. Compared to all institutions in our study, baccalaureate institutions are more likely to 
give an academic senate this approval authority. In terms of day-to-day operating responsibility, 
academic affairs officers, institutional researchers, and faculty members, in that order, are most 
likely to carry out day-to-day assessment activities. Whoever has this day-to-day operating 
responsibility most often reports to the chief academic officer. 

Evaluatio n of Assessment Process . Whether an institution has evaluated its assessment 
plan or process is often an indication of the importance it accords to student assessment as an 
institutional activity. Approximately half of baccalaureate institutions have evaluated their plan or 
policy for student assessment. All baccalaureate institutions should consider evaluating their plans 
and policies, in order to better understand whether their plan is meeting its objectives and whether 
the effort expended to assess students is benefiting the institution. Furthermore, our research 
found that evaluating the assessment plan and process leads to positive impacts of assessment 
information on faculty, students, and external relationships. 

Assessment Management Practices 

Assessment management practices are intentionally devised by institutions to manage their 
student assessment efforts. The four specific areas of practice we examined are: academic resource 
allocation, student information systems, internal access to student information, and distribution of 
assessment reports and studies. This section summarizes the assessment management practices 
used by baccalaureate institutions and provides suggestions regarding practices that maximize 
institutional uses and impacts of student assessment information. 

Academic Budget . Baccalaureate institutions as a whole are not allocating budgetary 
resources to academic units based on assessment engagement or results. If this practice was 
augmented, institutional leaders could demonstrate that they consider student assessment to be a 
high priority. However, our research found that allocating budgetary resources based on 
assessment engagement or results does not lead to positive impacts on faculty. It is likely that 
faculty members are not supportive of such budgetary practices. 

Computer Support . Currently, fewer than half of baccalaureate institutions report that they 
have a student information system that tracks students from application through graduation. Even 
fewer have either a system which includes student performance indicators or an integrated 
database. Along with improving the management of student assessment data, creating relational 
databases with student assessment information that can be linked to other organizational data 
should facilitate studying the link between students’ performance and their institutional 
interactions. Our research has found that institutions that conduct such studies report that their 
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student assessment information has been useful in making educational decisions and that this 
information has had an impact on both faculty and students. 

.Access to Assessment Information . Providing broad access to student assessment 
information is important if institutions want their constituents to make use of the information. 

Most baccalaureate institutions do provide internal access to student assessment information to 
senior academic administrators, institutional research professionals, department chairs, faculty 
advisors, and student affairs professionals. Our research found that providing such broad access 
to student assessment information increases the likelihood that this information will be used to 
make faculty-related decisions. 

Assessment Report Distribution . Not only is providing access to information helpful in 
making use of assessment information, but distributing reports of studies done using student 
assessment data is also important. The more people are aware of student assessment activities and 
results, the more likely they will be to use the information in their own decision making. While 
most baccalaureate institutions regularly distribute reports to traditional internal constituents such as 
faculty, academic administrators, and student affairs professionals, hardly any regularly distribute 
reports to students, the general public, or employers. Baccalaureate institutions should increase the 
number of constituents who receive their reports in order to increase the likelihood that the 
assessment data will be useful. 

Assessment Management Policies 

Assessment management policies refer to institutional policies devised to both support 
student assessment and to direct the use of student assessment information. Five content 
dimensions of assessment management policies were examined: student involvement in 
assessment; professional development on assessment for faculty and academic administrators; 
training in assessment for student affairs personnel; faculty evaluation and rewards; and academic 
planning and review processes. This section summarizes the assessment management policies 
used by baccalaureate institutions and provides suggestions regarding policies that maximize 
institutional uses and impacts of student assessment information. 

Student Involvement . While baccalaureate institutions report that many of their 
departments provide students with information about student assessment and require students to 
participate in assessment activities, they also report that only some departments provide students 
with feedback on their assessments. Providing individual feedback to students has the potential to 
help in improving their performance. Our research found that involving students in the assessment 
process led to greater use of assessment data in making educational decisions. If student 
involvement policies do not exist or do so only within a few departments, baccalaureate institutions 
should consider extending the breadth of these policies. 

Professi onal Development . While policies supporting professional development 
opportunities appear to be somewhat prevalent in baccalaureate institutions, activities that do not 
affect faculty time spent in the classroom are more prevalent than those that take faculty away from 
the classroom, such as paid leaves and course reductions. Offering a greater variety of 
professional development opportunities should increase the level of involvement in student 
assessment and also increase the likelihood that constituents will understand how to use student 
assessment information. Our research found that those institutions that offer professional 
development are more likely to use assessment data to make educational decisions. Baccalaureate 
institutions should increase their use of professional development so that internal constituents will 
understand the applications of student assessment information. 

Student Affairs Training . Baccalaureate institutions are less likely to have policies on 
providing assessment training to student affairs staff or administrators. When designing 
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professional development opportunities, baccalaureate institutions should not neglect these 
constituents. Involving student affairs staff in student assessment may lead to greater student 
involvement as the staff have opportunities for promoting assessment activities while they work 
with students. In fact, our research found that those institutions that provide student assessment 
training to their student affairs personnel are more likely to report that their assessment information 
has impacts on their students. They also report that they are more likely to use assessment 
information in making educational decisions. 

Faculty Evaluation and Rewards . Baccalaureate institutions rarely have policies that 
include assessment-related criteria in faculty evaluation and rewards. However, our research 
found that institutions that use such data to evaluate and reward faculty report that they also use 
student assessment information in making educational decisions and that this information has had 
an impact on their external relations. Perhaps external bodies expect to see that assessment data is 
used to make decisions about faculty. Institutions may understandably be reluctant to tie faculty 
evaluation and rewards to assessment results or indicators of student performance, but there are 
other options to consider. For example, institutions can use other criteria such as faculty 
scholarship on assessment, evidence of using assessment to improve teaching and learning, or 
participation in student assessment decision making. 

Academic Planning and Review . Another use for student assessment information is in 
planning and reviewing academic programs and courses. Most baccalaureate institutions are using 
student assessment information for this purpose. Our research found that institutions that use 
assessment information in academic planning and review are likely to report that their assessment 
information leads to positive impacts on faculty, students, and their external relationships. Those 
baccalaureate institutions that are not using assessment data in academic planning and review are 
encouraged to build in formal linkages between the analysis of student assessment data and specific 
internal decision making processes. 

Inventorying the Institutional Student Assessment Process 

Regardless of an institution’s history of and support for student assessment, it is important 
to take stock of what the institution is currently doing. The Institutional Framework presented in 
section two (Figure 1 of this monograph) provides a comprehensive perspective for such an 
examination. The ISSA inventory included as Appendix A provides a useful quasi-objective 
instrument for identifying the specific dimensions and activities associated with the student 
assessment process. 

Institutions are encouraged to identify a team of faculty and administrators most 
knowledgeable about and involved with student assessment to examine their institution’s activities 
on all the dimensions in the ISSA inventory. While most items are objective (e.g., which types of 
measures are being used and how extensively), the actual pattern on campus may not be widely 
known. Other items are more subjective (e.g., purposes for conducting student assessment) and 
can provide the basis for good discussion. 

This inventory of the institutional student assessment process then can be compared with 
national data for baccalaureate institutions which were presented in Appendix A and summarized in 
the previous sections of this monograph. This inventorying and comparison process may highlight 
activities, policies, and practices not currently used; identify areas needing greater attention; or 
focus on inconsistencies in patterns of activity (e.g. inconsistencies between stated purposes for 
student assessment and actual uses of student assessment data in academic decisions). 

For institutions with an extensive history of involvement with student assessment, such an 
inventory may serve as a useful basis for identifying new activities to be undertaken, for improving 
their existing activities and processes, for identifying issues or controversies that have been 
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avoided, or for redesigning processes that may not be worth the current expenditure of effort and 
resources. For institutions with less experience with student assessment, the inventory may 
identify existing activities on which to build or help focus attention in the institution on the 
importance of student assessment in improving institutional performance. In either type of 
institution, the inventory and self evaluation process should help both to focus faculty and 
administrative attention on the importance of viewing student assessment as an institutional process 
and to deal with it more systematically — linking the various domains of activity with institutional 
improvement. 



Planning for Student Assessment 

Student assessment in most higher education institutions has emerged, often sporadically, 
over the past decade due to the need to respond to an accreditation self study, a new state mandate, 
an academic administrator who promoted it, a faculty group who embraced it for their unit, or an 
institutional researcher or program review officer who was engaged in studies of student 
performance. While, according to the data in this report, some institutions are beginning to 
develop a plan or policy for student assessment or create a group responsible for it, there is little 
evidence of systematic planning that links the student assessment approach to external demands 
and to internal institutional governance and management patterns; develops the organizational and 
administrative activities, practices, and policies to support it; and then uses the data for academic 
decisions and monitors their impact. Clearly there are significant institutional differences and 
complex issues to be addressed if student assessment is to have positive effects. These deserve 
some systematic, planned attention. The results of an institutional inventory can provide the basis 
for such an effort. 

While we do not advocate a cookbook or standardized approach to planning for student 
assessment (each institution needs to design its own planning approach to reflect its own 
governance and leadership styles and traditions), the Institutional Framework (Figure 1 in section 
two) and the institutional inventory provide a useful basis for redesigning or planning a student 
assessment process. Using that self-assessment, the following are planning issues that need to be 
addressed: 

External Influences 

1 . What is the nature of our state assessment process? How is it formulated and what are 
its requirements/implications for us? 

2 . What are the accreditation requirements for our institutional and key professional 
accreditation bodies? 

3 . What do some of our primary external constituents expect of our graduates? 

4 . What are sources of support (educational, financial, technical) for our student 
assessment efforts? 

Institution-Wide Support Patterns 

j 

1 . What emphasis is placed on student assessment in our mission statements? What are the 
intended purposes of our institution’s assessment efforts? 

2 . What institution-wide administrative, governance, and academic activities support 
student assessment? Who are key leadership support groups and how are they 
involved? 
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3 . 



What is the nature of our institution-wide plan or policy for student assessment? The 
role and membership of a coordinating or planning body? And the pattern of authority 
and responsibility for administering the process? 

4. Is the student assessment process to be evaluated? By whom? Using what criteria? 

Approaches to Student Assessment 

1 . Should we have an institution-wide or decentralized (by academic unit) approach to 
student assessment? 

2 . What types of measures are to be used? How widely? At what points in time? 

3 . What types of instruments and methods are appropriate? What technical or professional 
support does that require? 

4. What studies of the influence of students’ educationally related experiences on their 
performance are to be done? What reports of student performance are to be prepared 
and distributed? 

Assessment Management Practices and Policies 

1 . What assessment management practices exist to guide student assessment? (e.g., 
resource allocation, information systems, data access, and report distribution practices.) 

2. What institutional policies promote the use of student assessment? (e.g., policies on 
student involvement, professional development, student affairs training, faculty 
evaluation, and academic planning.) 

Uses and Impacts of Student Assessment 

1 . How will we assure use of student assessment information on educationally- and 
faculty-related academic decisions? 

2 . How will we monitor the impact of student assessment on our students, faculty, 
academic and instructional patterns, and on our external relationships? 

While planning for student assessment may not resolve all of these questions, they should 
be addressed as should issues of the balance between the effort and resources required to maintain 
the institution’s student assessment process and the educational and institutional benefits. When 
planning is combined with inventorying, these two processes can become a powerful mechanism 
for understanding and improving institutional student assessment endeavors. Inventorying 
existing student assessment processes is a first step toward understanding the nature and extent of 
an institution’s student assessment activities. This can be followed by a planning process that 
considers the recommendations for baccalaureate institutions that were presented in this section. 
Using a systematic planning approach should increase the likelihood that student assessment will 
contribute to improved institutional performance. 
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Appendix A 



Inventory of 
Institutional Support for 
Student Assessment for 
Baccalaureate Institutions 
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Inventory of 
Institutional Support for 
Student Assessment for 
Baccalaureate Institutions 



For The Research Program on 
Institutional Support for Student Assessment 




NCPI 

1 National 
Center for 
Postsecondary 
Improvement 



NCPI - Project 5.2 
University of Michigan 
Ann Arbor, Michigan 48109-1259 
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An Introduction to the ISSA 



The Institutional Support for Student Assessment Inventory (ISSA) was developed as part of a national research 
program examining the Organizational and Administrative Support for Student Assessment for the National Center 
for Postsecondary Improvement (NCPI). The ISSA is designed as an institutional inventory of the organizational 
and administrative practices that have been designed and implemented to support the use of student assessment on 
your campus. 

Institutional Support Practices are those organized activities, policies, and procedures that your institution has 
intentionally designed to enhance the practice of student assessment. Student Assessment refers to those activities 
focused on measuring dimensions of student performance other than traditional end of course grading. 

This national survey is designed to identify institutional support practices for undergraduate student assessment. 

The project also examines the factors influencing the adoption of various support practices and how those practices 
enhance the impact of student assessment for institutional improvement. 

We understand that being selected for this survey will require a commitment of time to complete and we appreciate 
your involvement. This instrument is also intended as an institutional self-assessment inventory to facilitate 
examination of your institution’s own organizational and administrative practices which support student assessment. 
We encourage each institution to use the survey in this manner. You will receive a summary report of survey 
responses to all compare with your own institutional profile. 

Completing the ISSA 

The main purpose is to obtain a profile of your institution’s current approach to undergraduate student assessment 
and its support practices. The inventory may be completed by one individual or group of individuals who are most 
familiar with the patterns of undergraduate student assessment on your campus. It should take less than one hour to 
complete. 

• Please keep in mind that the questions refer to undergraduate education at your institution. 

• Respond to each item in the questionnaire to the best of your knowledge. 

The questionnaire is coded to allow follow up only. Individual institutions will not be identified in any analyses or 
reports. 

Return the completed questionnaire in the enclosed return envelope. Any questions concerning the survey can be 
addressed to the following: 



National Center for Postsecondary Improvement Project 5.2 
School of Education 
University of Michigan 
610 E. University, Room 2339 
Ann Arbor, MI 48109-1259 
Phone: 734-647-2464 
Fax: 734-936-2741 
Email: ncpi.proj52@umich.edu 



Marvin W. Peterson, Project Director 

NCPI is funded by the U.S. Department of Education’s Office of Educational Research and Improvement under 
OERI grant number R309A60001 

©1997, The Regents of the University of Michigan 
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1. Institutional Approach to Student Assessment 



A. Type, Extent and Timing of Student Assessment 

We are interested in your institution’s routine practices of collecting different types of undergraduate student 
performance data, the extent to which they are collected, and when they are collected. For each of the following 
content types of undergraduate student performance data: 

1) indicate the extent to which each type is collected 

2) for each type of data collected, check whether it is collected at entry, during enrollment, at exit, or 



a combination of these data collection points. 










TvDe 


Extent 






Timing 






Collected 


Collected 


Collected 


Collected 


Collected 


Collected 


Currently Enrolled Students 


Not for some 

Collected students 


for many 
students 


for all 
students 


at 

entry 


while 

enrolled 


at 

exit 




(circle cue number for each 


item) 


(check all that apply for each item) 




1 2 


3 


4 










BAC 


All 










1. Student academic intentions or 














expectations 

2. Basic college-readiness 


3.33 


3.25 










skills (reading, writing, 
mathematics, etc.) 


3.35 


3.44 










3. Higher-order skills (critical 
thinking, problem solving) 

4. General education 


2.41* 


2.10 




— 


— 


— 


competencies 

5. Competence in major field 


2.76 


2.55 










of study (discipline- or 
program-specific knowledge) 


2.92* 


2.60 










6. Vocational or professional 
skills 

7. Personal growth and 


1.97 


2.11 




— 


— 


— 


affective development 
(values, attitudes, social 
development, etc.) 


2.51* 


2.12 










8. Student experiences and 
involvement with institution 

9. Student satisfaction with 


2.79 


2.57 




— 


■ 


— 


institution 


3.13 


2.96 










10. Student academic progress 














(retention, graduation rates) 


3.87 


3.76 










Former Students 














1 1. Vocational or professional 














outcomes (career goals, job 
attainment or performance) 


2.74 ' 


2.72 










12. Further education 














(transfer, degree attainment, 
graduate study) 


2.74 


2.69 










13. Civic or social roles 














(political, social or 
community involvement) 


2.26* 


1.80 










14. Satisfaction and experiences 














with institution after leaving 


2.63 


2.63 
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B. 



Student Assessment Instruments 



Does your institution employ institutionally or externally developed instruments or tests for the following types of 
undergraduate student assessment information? (circle £ll that apply for each item): 

Source of Instrument 





Not 


Institutionally 


State 


Commercially 


Content of Instrument 


used 

1 


developed 

2 


provided 

3 


available 

4 


1. Student plans, goals, or expectations 


BAC: 21% 


46% 


0.3% 


42%* 




All: 21% 


51% 


4% 


32% 


2. Basic college-readiness skills 


BAC: 12% 


50% * 


3% 


58% 


(reading, writing, mathematics, etc.) 


All: 6% 


38% 


11% 


67% 


3. Higher-order skills 


BAC: 39% 


30% 


2% 


37% 


(critical thinking, problem solving) 


All: 42% 


29% 


2% 


32% 


4. General education competencies 


BAC: 28% 


45% 


7% 


37% 




All: 32% 


40% 


5% 


33% 


5. Competence in major field of study 


BAC: 11% 


74%* 


9% 


51 %* 


(discipline- or program-specific knowledge) 


All: 19% 


64% 


12% 


39% 


6. Vocational or professional skills (excluding 


BAC: 43% 


33%* 


15% 


23% 


licensure exams) 


All: 34% 


43% 


14% 


25% 


7. Personal growth and affective development 


BAC: 27%* 


48% 


2% 


41% 


(values, attitudes, social development, etc.) 


All: 38% 


39% 


2% 


30% 


8. Student effort, experiences or involvement 


BAC: 16% 


66% 


1% 


35% * 


with institution 


All: 21% 


60% 


4% 


24% 


9. Student satisfaction with institution 


BAC: 6% 


72% 


3% 


43% 




All: 4% 


73% 


8% 


35% 


10. Alumni satisfaction and experiences 


BAC: 7% 


81% 


7% 


23% 




All: 10% 


78% 


8% 


15% 
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c. 



Other Student Assessment Methods 



To what extent does your institution use the following methods to collect undergraduate student assessment 
information? (circle one number for each item): 



Other Student Assessment Methods 


Not 


Used in 


Used in Used in 




used 


some units' 


most units all units 




1 


2 


3 4 






BAC 


All 


1. Observations of student performance 








(simulations, demonstrations, lab) 




2.34 


2.26 


2. Student portfolios or comprehensive projects 




2.29 


2.10 


3. Student performance in capstone courses 




2.50* 


2.15 


4. Student interviews or focus groups 




1.96 


1.84 


5. Transcript analysis 




2.24 


2.16 


6. External examination of students 








(licensure exams, external reviewers) 




1.98 


2.02 


7. Special surveys of or interviews with 








withdrawing students 




2.78* 


2.40 


8. Alumni interviews or focus groups 




1.95 


1.90 


9. Employer interviews or focus groups 




1.66 


1.87 


D. Student Sub-Populations 








Does your institution use different assessment methods for the following sub-populations of undergraduate students? 


(check fine for each item): 


Different 




Same as Other Students 




BAC 


All 




1. Adult students 


17% 


10% 




2. Part-time students 


7% 


5% 




3. Minority students 


1% 


2% 




4. Distance education students 


22% 


22% 





* “Unit” refers to academic areas such as departments, divisions, schools, or colleges. 
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E. Student Assessment Studies 



Does your institution conduct studies of the relationship between the following experiences and students’ 
performance (check all that apply): 

1 . Student course-taking patterns 

2. Exposure to different instructional or teaching methods 

3. Patterns of student-faculty interaction 

4. Extra-curricular activities 

5. Residence arrangements 

6. Student financial aid and/or concurrent employment 

7. Admission standards or policies 

8. Academic advising patterns 

9. Classroom, library and/or computing resources 



F. Student Performance Profiles or Reports 



levels of aggregation (check all that apply): 

1. Institution wide 

2. Schools or colleges 

3. Academic programs or departments 

4. Special populations or subgroups/students 

5. By course or groups of courses 

6. Do not provide any reports 



BAC 


All 


23% 


26% 


16% 


21% 


16% 


14% 


30% 


24% 


27% 


21% 


27% 


30% 


50% 


42% 


27% 


26% 


17% 


17% 


and student performance 34% 


38% 


tudent performance information at the following 


BAC 


All 


69% 


69% 


19%* 


31% 


64% 


65% 


37% 


46% 


29% 


36% 


9% 


11% 



LLInstitutional Support for Student Assessment 



A. Institutional Emphasis 



1 . Your institutional mission statement explicitly (check all that apply): 

BAC All 

a. emphasizes excellence in undergraduate education as an institutional priority 88% 82% 

b. identifies the educational outcomes intended for your students 61 % 52% 

c. refers to student assessment as an important institutional activity 16% 19% 

d. does not explicitly mention any of the above 7 % \\% 



2. For how many years has your institution engaged in student assessment? 
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B. Purpose of Student Assessment 

The following are often intended purposes of an institution’s undergraduate student assessment process. Please rate 
the importance of each for your institution, (circle one number for each item); 



Purpose 


No 


Minor 


Moderate 


Very 


Importance 


Importance 


Importance 


Important 




1 


2 


3 


4 






BAC 


All 




1. Preparing institutional self-study 










for accreditation 




3.63 


3.86 




2. Meeting state reporting 
requirements 

3. Guiding internal resource 




2.30* 


2.89 




allocation decisions 




2.62 


2.71 




4. Guiding undergraduate academic 
program improvement 

5. Improving the achievement of 




3.51 


3.43 




undergraduate students 




3.47 


3.48 




6. Improving faculty instructional 
performance 

7. Other (briefly describe): 




3.08 


3.02 





C. Administrative and Governance Activities 

Institutions have introduced a variety of administrative or governance activities that address or promote student 
assessment. Does your institution engage in any of the following activities? (check all that apply): 



Annual presidential or other institution-wide initiatives, forums or seminars 


BAC 


All 


on assessment 

Rewards or incentives for academic and student affairs administrators who 


33% 


41% 


promote use of student assessment in their unit 

Incentives for academic units to use student assessment information in 


6% 


6% 


their evaluation and improvement efforts 


25% 


27% 


Student assessment workshops for academic and student affairs administrators 


49% 


56% 


Board of trustees committee that addresses student assessment 


14% 


13% 


Faculty governance committee that addresses student assessment issues 


69%* 


58% 


Student representation on student assessment committees 


37% 


33% 



D. Support for Student Assessment 

Use the scale below to rate the degree to which various groups within your institution support undergraduate student 
assessment activities (circle one number for each item): 



Very 

Unsupportive 

1 

1 . Board of trustees 

2. Chief executive officer 

3. Academic affairs administrators 

4. Student affairs administrators 

5. Faculty governance 

6. Students 



Somewhat 


Neutral, 




Very 


Unsupportive 


Unknown 


Supportive 


Supportive 


2 


3 


4 


5 


BAC 


ALL 






3.79 


3.84 






4.39 


4.41 






4.69 


4.64 






4.33 


4.33 






3.83 


3.80 






3.33 


3.33 







E. Planning and Coordinating Student Assessment 



1 . Which of the following best describes your institution’s plan or policy for undergraduate student 
assessment? Your institution (check ail that apply): 



a. 



b. 



c. 



d. 

e. 

£ 

g. 



has a formally adopted institutional plan or policy requiring specified 
undergraduate student assessment activities of all academic units or 
programs 

has a formally adopted plan or policy for undergraduate student assessment 
in som^ academic units or program areas (e.g. general education or 
academic majors) 

has a formally adopted institutional plan or policy requiring all academic 
units or programs to develop their own undergraduate student assessment 
plan 

has a formally adopted institutional plan or policy stipulating institution- 
wide activities to be conducted by a central committee, office, or officer 
has no formal plan or policy but academic units or programs are encouraged 
to conduct their own undergraduate student assessment activities 
is currently developing a plan or policy for undergraduate student assessment 
does not have an undergraduate student assessment plan or policy 
(SKIP TO E-6) 



BAC All 



55 % 


50% 


14% 


19% 


46% 


39% 


40% 


38% 


14% 


13% 


17% 


17% 


2% 


4% 



2. Is there an institution-wide group (committee, task force, etc.) that is primarily responsible for ongoing 
planning and policy setting for undergraduate student assessment? (check one) : 



BAC All 

a. yes 730/0 70% 

b. no (skip to Question E-5) 



er!c 
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3. If yes, who serves on this group? (check ah that apply): 



a. 


Chief executive officer 


BAC 

12% 


All 

13% 


b. 


Academic affairs administrator(s)/stafF 


85% 


86% 


c. 


Student affairs administrator(s)/staff 


40% 


54% 


d. 


Institutional research administrator(s)/stafF 


53% 


61% 


e. 


Academic review and evaluation administrator(s)/staff 


20% 


24% 


f 


Student assessment administrator(s)/stafF 


22 %* 


32% 


g* 


Faculty 


90% 


91% 


h. 


Students 


38% 


33% 


i. 


Other 


15% 


12% 



Who has executive responsibility for or who chairs the institution-wide group responsible for the ongoing 
planning or policy-setting process for undergraduate student assessment? (check all that apply): 







BAC 


All 


a. 


Academic affairs administrator 


60% 


55% 


b. 


Student affairs administrator 


3% 


7% 


c. 


Institutional research officer 


19% 


18% 


d. 


Academic review and evaluation officer 


5% • 


5% 


e. 


Student assessment officer (if separate) 


10% 


8% 


f 


Faculty member 


28% 


31% 


g- 


Other 


7% 


11% 



5. Who approves any changes in your institution’s plan or policies for undergraduate student assessment? 
(check all that apply): 







BAC 


All 


a. 


Board of trustees 


12% 


17% 


b. 


Chief executive officer 


40% 


45% 


c. 


Chief academic affairs officer 


76% 


75% 


d. 


Chief student affairs officer 


13% 


20% 


e. 


Institutional research officer 


21% 


18% 


f 


Academic review and evaluation officer 


7% 


8% 


g. 


Student assessment officer 


9% 


10% 


j- 


Student government 


0.3% 


1% 


h. 


Academic senate or other faculty committee(s) 


52%* 


39% 


i. 


Faculty union 


7% 


4% 


k. 


Other 


15% 


14% 



6. Who has operational responsibility for your institution’s day-to-day undergraduate student assessment 
activities (e.g., instrument development, data collection, analysis, and reporting)? (check all that apply): 







BAC 


All 


a. 


Academic affairs administrator 


54% 


45% 


b. 


Student affairs administrator 


14% 


20% 


c. 


Institutional research officer 


42% 


45% 


d. 


Academic review and evaluation officer 


10% 


9% 


e. 


Student assessment officer 


13% 


15% 


f 


Faculty members) 


39% 


33% 


g- 


Other 


10% 


13% 


h. 


No one (SKIP TO QUESTION E8) 


3% 


3% 
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7. To whom does the individual with operational responsibility for day-to-day student assessment activities 
directly report? (check one): 







BAC 


All 


a. 


Chief executive officer 


28% 


29% 


b. 


Chief academic officer 


66% * 


56% 


c. 


Chief student affairs officer 


2% 


7% 


d. 


Institutional research officer 


2% 


3% 


e. 


Academic review and evaluation officer 


2% 


2% 


f 


Other 


6% 


10% 



8. Is there an office which provides faculty consultation in using student assessment for instructional 



improvement or curriculum development? (check onek 








BAC 


All 


a. yes 


41% 


47% 


b. no 


59% 


53% 



9. If yes, what is the name of the office? 

F. Evaluating Your Institution’s Student Assessment Plan or Process 



Has your institution evaluated its undergraduate student assessment process? 


(check oneV 








BAC 


All 


a. 


yes, with a formal evaluation 


21% 


22% 


b. 


yes, with an informal evaluation 


26% 


27% 


c. 


currently developing evaluation plans (SKIP TO SECTION III) 


33% 


29% 


d. 


not currently evaluating or planning to evaluate assessment process 
(SKIP TO SECTION) 


21% 


21% 



2. In evaluating your institution’s student assessment process, which of the following elements of that 
process were reviewed? (check all that apply): 



a. 


your student assessment plan and policies 


BAC 

90% 


All 

81% 


b. 


the structure and responsibility for student assessment 


73% 


64% 


c. 


achievement of your institution’s intended objectives for 
student assessment 


76% 


70% 


d. 


reliability and validity of student assessment instruments and methods 


52% 


54% 


e. 


quality of data analysis 


55% 


51% 


£ 


use of student assessment information in institutional decision-making 


73% 


66% 


8- 


the problems encountered while conducting student assessment activities 


74% 


69% 


h. 


comparison of the costs and benefits of student assessment 


31% 


22% 
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111. External Influences on Institutional Student Assessment Activities 



A. State Role (FOR STATE-FUNDED INSTITUTIONS ONLY; ALL OTHERS SKIP TO QUESTION HI. B-l) 
1 . Was your state’s plan/requirement for student assessment primarily developed (check oneY ): 







BAC 


All 


a. 


by state-level officials 


5%* 


16% 


b. 


through joint consultation between state officials and institutional representatives 24%* 


39% 


c. 


no statewide plan or requirement for student assessment exists (SKIP TO III. B-l) 71 %* 


46% 


State requirements for student assessment (check ail that apply): 










BAC 


All 


a. 


were an important reason for your institution to initiate undergraduate 








student assessment 


30 %* 


45% 


b. 


have increased your institution’s involvement in undergraduate student 








assessment 


42 %* 


62% 


c. 


have not been a factor in your institution’s undergraduate student assessment 








activities 


47%* 


22% 


d. 


have been a negative influence on your institution’s undergraduate student 








assessment activities 


0% 


4% 


Your state’s reporting requirements include (check all that apply): 










BAC 


All 


a. 


evidence that a student assessment plan is in place 


88%* 


68% 


b. 


measurement of state-mandated student performance indicators 


56% 


64% 


c. 


institutionally-devised student performance indicators 


59%* 


49% 


d. 


evidence of institutional use of student assessment information 


74%* 


52% 



4. How has your state higher education agency reviewed or evaluated your institution’s undergraduate student 
assessment plan or process after it was implemented? (check ail that apply): 







BAC 


All 


a. 


reviewed by state officials 


45% 


42% 


b. 


reviewed using external reviewers 


8% 


16% 


c. 


required an institutional self-review 


23% 


24% 


d. 


no post hoc review has occurred (SKIP TO QUESTION B- 1 ) 


50% 


44% 



5. The state review of your institution’s undergraduate student assessment plan or process included (check all 
that apply): 



a. 


review of your institution’s student assessment process itself 


BAC 

76% 


All 

67% 


b. 


comparison of your institution’s student performance record with 
your past performance 


43% 


44% 


c. 


comparison of your institution’s student performance record with- 








peer institutions 


14%* 


36% 


d. 


comparison of your institution’s student performance record with 
institutions in your state 


29% 


38% 


e. 


other (briefly describe) 


24%* 


10% 
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B. Regional Accrediting Role in Student Assessment 



Has your institution gone through a regional self study accreditation review which required undergraduate 
student assessment? (check one) : 







BAC 


All 




a. yes 


81% 


80% 




b. no 


19%* 


29% 


Regional accreditation agency requirements for undergraduate student assessment (check all that apply): 




were an important reason for your institution to initiate undergraduate student 


BAC 


All 


a. 






b. 


assessment 


65% 


64% 


have increased your institution’s involvement in undergraduate student 








assessment 


85% 


79% 


c. 

d. 


have not been a factor in your institution’s undergraduate student assessment 
activities 


8% 


12% 


have had a negative influence on your institution’s undergraduate std. 
assessment activities 


0 . 3 % 


0.9% 





Your institution’s regional accreditation agency requires (check &U that apply): 



a. 


evidence that a student assessment plan or process is in place 


BAC 

92% 


All 

90% 


b. 


intended institutional uses of student assessment information 


75% 


73% 


c. 


results of student assessment 


66 % 


66% 


d. 


evidence of actual institutional use of student assessment information 


79% 


77% 


e. 


unfamiliar with regional accreditation requirements for student assessment 


4% 


5% 
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C. External Sources of Support for Assessment 



I . Has your institution received external grants to improve undergraduate student assessment practices from any of 
the following? (check all that apply): 







BAC 


All 


a. 


FIPSE 


8% 


6% 


b. 


other federal agencies (please identify): 


5% 


7% 


c. 


a state incentive program 


3% 


7% 


d. 


private foundations or corporate sources (please identify): 


11% 


6% 


e. 


no known external grants received 


79% 


79% 



2. Has your institution used any of the following student assessment services offered by the following 
postsecondary organizations? (check all services that apply for each type of organization): 



Student Assessment Service Used 



Type of Postsecondary Organization 


Not used or 


Consultation 


Assessment 


Training 


Publications 
or research 




not available 


services 


conferences 


workshops 


reports 


a. Professional associations (Institutional, 


BAC: 30% 


11% 


52% 


30% 


50% 


disciplinary, or administrative) 


All: 29% 


13% 


51% 


32% 


51% 




BAC: 26% 


17% 


44% 


33% 


44% 


b. Regional accrediting association 


All: 30% 


19% 


41% 


32% 


45% 




BAC:67%* 


11% 


15%* 


14% 


18% 


c. State-level agency 


All: 54% 


14% 


26% 


22% 


22% 




BAC: 48% 


17% 


36% 


19% 


27% 


d. Consortium of institutions 


All: 53% 


13% 


30% 


18% 


20% 



IV. Academic Management Policies and Practices for Student Assessment 

Institutions have a wide array of formally organized policies, activities, and procedures intended to enhance or 
support the collection and use of undergraduate student assessment information. The following policies and 
practices have been identified in many institutions. 

For Questions a through D, indicate whether the following policies or practices exist at your 

INSTITUTION. 



A. Resource Allocation for Student Assessment (check all that apply): 





BAC 


All 


An explicit operating budget allocation is made-to support student assessment. 


57% 


49% 


An academic budget process that considers student performance indicators in 
resource allocation to academic units. 


15% 


23% 


An academic budget process that compares academic units on student performance 
indicators and allocates resources competitively. 


2% 


2% 


An academic budget process that rewards academic units for improvement based 
on their own past student performance indicators. ~ ~ 


3% 


3% 




79 



88 



B. Student Assessment Information System (check all that apply): 



1. Key student assessment activities have been scheduled into the academic calendar. 


BAC 

64% 


All 

57% 


2. A computerized student information system which includes student performance 
indicators. 


19% 


28% 


3. Student information system tracks individual students from application through 
graduation. 


40% 


42% 


4. Student assessment database integrated with faculty, curricular, and financial 
databases. 


8% 


10% 



C. Access to Individual Student Assessment Information (check all that apply): 
Student assessment information on individual students is available to: 





BAC 


All 


1 . Institutional research, assessment or evaluation professionals 


72% 


76% 


2. Senior academic administrators 


75% 


72% 


3. Department chairs or academic program administrators 


71% 


73% 


4. Student affairs professionals 


51% 


58% 


5. Faculty advisors 


66% 


66% 



D. Distribution of Student Assessment Reports and Studies (check all that apply): 
Student assessment reports and studies or appropriate summaries are regularly distributed to: 





BAC 


All 


1. Students 


20% 


19% 


2. Faculty 


71% 


67% 


3. Academic administrators 


91% 


86% 


4. Student affairs professionals 


51% 


58% 


5. Employers 


2% 


5% 


6. The general public 


7% 


8% 



For Questions E through H, use the following scale to indicate the extent to which each of the 

FOLLOWING POLICIES AND PRACTICES EXIST AT YOUR INSTITUTION (Circle one number for each item). 



E. Student Policies on Student Assessment 



1. Students are required to participate in student 
assessment 

activities 

2. Incentives are provided to encourage students to 
participate in student assessment activities - 

3. Information regarding the purpose and uses of student 
assessment is provided to students 

4. Students are provided with individual feedback 
regarding 

their own student performance results 



Not 


Done 


Done in 


Done in 


Done 


done 


in 


some 


many 


in 


at all 


a few 


depts. 


depts. 


most 


1 


depts. 


3 


4 


depts. 




2 






5 




BAC 




All 




- 


4.02 




3.77 






-2.06 


• : 


1.87 






3.72 




3.52 






3.25 




3.21 
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F. Professional Development 



1 . Faculty are required to learn about or receive training on 
student assessment 


BAC 

2.48 


All 

2.47 


2. Funds for faculty to attend or present at professional 
conferences on student assessment are available 


3.08 


3.08 


3. Workshops, seminars, or consultative services for 
faculty on the use of student assessment in course 
design or instruction are offered 


2.71 


2.90 


4. Assistance for faculty in the form of paid leaves, 
stipends, mini grants or course reduction to improve 
use 


1.89 


2.00 


of student assessment is provided 
5. Workshops and seminars for department chairs, deans, 
and other academic administrators to improve use of 
student assessment in their unit is provided 


2.39 


2.55 


6. Student affairs staff are required to learn about or receive 
training related to student assessment 


1.94 


2.22 


7. Student assessment workshops for student affairs 
administrators are provided 


1.87* 


2.22 



G. Faculty Evaluation and Rewards 



1 . Faculty evaluation for promotion considers evidence of 
student performance in their classes (not just student 
teaching evaluation) 


BAC 

2.05 




All 

1.84 


2. Faculty evaluation for annual salary and merit increases 
incorporates evidence of student performance 


1.67 




1.56 


3. Faculty scholarship on or innovative uses of student 
assessment is considered in promotion, tenure, or salary 
reviews 


2.27 




2.01 


4. Faculty willingness to use or to participate in student 
assessment activities is considered in faculty promotion, 
tenure, or salary reviews 


2.35* 




1.99 


5. Faculty receive public recognition or awards for 
innovative or effective use of student assessment 


1.58 




1.58 


6. Faculty hiring process considers experience or skill in 
student assessment 


1.66 


■) 


1.68 


7. Faculty are encouraged to assess student learning in 
theirclasses 


4.12 




3.99 
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Not 


Done 


Done in 


Done in 


Done 




done 


in 


some 


many 


in 


H. Academic Planning and Review 


at all 


a few 


depts. 


depts. 


most 


1 


depts. 


3 


4 


depts. 

c 


Your institution incorporates student performance data into 
the following processes: 




z 












BAC 




All 




1 . Academic department or undergraduate program 
planning 




3.72 




3.67 




or review 












2. General education or core curriculum review 




3.72 




3.55 




3. Course-level review and development 

4. Review and planning for student academic support 




3.28 




3.36 




services 




3.07 




3.09 





V. Impacts of Student Assessment 

A. Decision Making 

To what extent has the use of information available from your undergraduate student assessment process influenced 
the following actions? (circle one number for each item): 





No action or 


Action taken, 


Action taken, 


Action taken, 


Institutional Actions 


influence 


data not 


data somewhat 


data very 




unknown 


influential 


influential 


influential 




1 


r . 2 


3 


4 






BAC 


All 




1. Revising your undergraduate 
academic mission or goals 

2. Designing or reorganizing 




2.09 


2.06 




academic programs or majors 
3. Designing or reorganizing student 




2.61 


2.54 




affairs units 

4. Allocating resources to academic 




1.93 


1.91 




units 




1.77 


1.81 




5. Modifying student assessment 
plans, policies, or processes 

6. Deciding faculty promotion and 




1 . 55 * 


2.61 




tenure 


- 


1.70 


1.46 




7. Deciding faculty salary increases 










or rewards (release time, travel 
funds, etc.) 




1.49 


1.39 




8. Revising or modifying general 










education curriculum 




2.57 


2.47 




9. Creating or modifying student 








i 


out- of-class learning experiences 
(e.g. internships, service learning) 




2.34 


2.14 




10. Creating or modifying distance 
- learning initiatives 

11. Modifying instructional or 




1.52 - 


- 1.72 




teaching methods 
12. Modifying student academic 




2.43 


2.47 




support services (e.g. advising, 
tutoring) 




2.49 


2.56 
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B. Institutional Impacts 



Have you monitored the following institutional indicators and been able to document the impact of student 
assessment information on them? (circle one number for each item): 





Not monitored. Monitored, 


Monitored, 


Monitored, 


Internal Impacts 


do not negative 


no known 


positive 


know impact 


impact 


impact 




1 2 


3 


4 




BAC 


All 




1 . Affected campus discussions of 
undergraduate education 


2.57 


2.28 




2. Contributed to faculty 








satisfaction 


1.88 


1.69 




3. Contributed to faculty interest 
in teaching 

4. Led to changes in instructional 


1.98 


1.88 




or teaching methods used 


2.60 


2.45 




5. Contributed to student 








satisfaction 


2.11 


2.03 




6. Affected student retention or 
graduation rates 

7. Affected student grade 


2.26 


2.20 




performance 

8. Affected student achievement on 


1.91 


1.95 




external examinations (e.g. 








professional licensure, GRE) 


1.99 


1.97 




External Impacts 


BAC 


All 




9. Affected student applications or 








student acceptance rates 


1.63 


1.48 




10. Affected allocation or share of 








state funding 


1.24 


1.46 




1 1. Affected evaluation from 








regional accreditation agency 
12. Affected private fund-raising 


2.57 


2.55 




results 


1.65 


1.42 




13. Affected success on grant 








applications 


1.75 


1.65 




14. Affected communication with 








external constituents 


1.87 


1.75 




15. Affected institutional reputation 








or image 


2.04 


1.94 





Note: * denotes difference greater than or equal to 3 or 10% from average of all institutions. 
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VI. Further Studies - Optional 



This page will be removed from the questionnaire before it is processed and completion of it is optional. However, 
we would like to know more about your institution’s experience with student assessment and we would like to be 
able to respond to you personally with a follow up report. 



Within the next year several institutions will be invited to participate in a more intensive study of the impacts of 
their student assessment practices and policies. Would you be interested in participating in a case study? 

yes 

possibly 

no 



If you are interested, we would appreciate any additional information regarding your student assessment practices 
that you believe would be of interest to other institutions. If you believe your approach to student assessment or its 
impacts are unusual, please describe it briefly (or enclose a report you think captures your experience). 



Please provide your name and address if you are interested in receiving a personal summary report of this survey. 

Name: 

Title: 

Institution: 

Address: 



Phone: 

E-Mail: 



Thank you for taking the time to complete this instrument. 



Appendix B 



Construction and 
Content of Variables 



0 

ERIC 



85 



94 



Variable 



Definition 



Institutional Characteristics 

Enrollment Single item. Reflects number of students enrolled in institution. Data from 

1995 Integrated Postsecondary Education Data System (IPEDS). 

Control Single item. (1 = public; 0 = private). Data from IPEDS. 

Institutional type Four dummy-coded single items. Reflects the institution’s camegie type. 

(Associate of Arts, Baccalaureate, Doctoral, and Research. Master’s institutions 
was the omitted category.) Data from IPEDS. 

External Influences on Student Assessment 

State initiative Single item. Reflects whether the state’s assessment initiatives were guided by 

legislative or other means (1 = no state plan; 2 = state policy; 3 = state statute; 4 
= combination of policy & statute): Data from SAS. 



State approach Single item. Reflects whether states mandate common indicators and outcomes 

(1 = no indicators or outcomes; 2 = institutional specific; 3 = common for some; 
4 = common for all). Data from SAS. 

Accrediting association Five dummy-coded single items. Reflects the institution’s regional accreditation 

association membership (Middle States; North Central; New England; Southern; 
Western. Northwest region was the omitted region). Data from IPEDS. 

Development of state plan Single item. (Ill A 1) Reflects how state plan for student assessment was 

primarily developed (1 = state; 2 = joint consultation between state and 
institution; 3 = no state plan or requirement). 

State influence Four single items. (Ill A 2 a-d) Reflect the influence of state requirements on the 

institutions assessment activities: a = important reason to initiate student 
assessment; b = increased institution’s involvement in assessment; c = have not 
been a factor in assessment activities; d = have been negative influence on 
assessment activities (I = yes; 0 = no). 



State reporting requirements Four single items. (Ill A 3 a-d) Reflect the state’s reporting requirements: a = 

evidence that assessment plan is in place; b = measurement of state mandated 
indicators; c = use of institutionally devised indicators; d = evidence of 
institutional use of assessment information (1 = yes; 0 = no). 

State review methods Four single items. (Ill A 4 a-d) Reflect the method used by state to review the 

institutions assessment activities: a = reviewed by state officials; b = reviewed 
using external reviewers; c = required institutional self-review; d = no review 
occurred ( 1 = yes; 0 = no). 

State review criteria Five single items. (Ill A 5 a-e) Reflect the processes included in the state review 

of the institutions assessment activities: a = review of institutions process itself; 
b = compare student performance record with past record; c = compare student 
performance record with peer institutions; d = compare student performance record 
with other in state; e = other (1 = yes; 0 = no). 
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Definition 



Variable 

Accrediting influence 



Accrediting reporting 
requirements 



External sources of support 



Use of external services 



Institutional Approach to 
Academic intentions 



Basic college-readiness skills 
Cognitive assessment 



Affective assessment 



Academic progress 



O 




Four single items. (Ill B 2 a-d) Reflect the influence of regional accreditation 
agency requirements on the institutions assessment activities: a = important 
reason to initiate student assessment; b = increased institution’s involvement in 
assessment; c = have not been a factor in assessment activities; d = have been 
negative influence on assessment activities (1 = yes; 0 = no). 

Five single items. (Ill B 3 a-e) Reflect the regional accreditation agency 
reporting requirements: a = evidence that assessment plan is in place; b = 
intended uses of assessment information; c = results of assessment; d = evidence 
of actual institutional use of assessment information; e = unfamiliar with regional 
accreditation requirements (1 = yes; 0 = no). 

Five single items. (Ill C 1 a-e) Reflect the sources of support received to 
improve student assessment practices: a = FIPSE; b = other federal agencies; c = 
state incentive program; d = private foundation or corporate source; e = no known 
external grants (1 = yes; 0 = no). 

Four single items. (Ill C 2 a-d) Reflect the use of services offered by each of the 
following type of postsecondary organization: a = professional associations; b = 
regional accrediting association; c = state-level agency; d = consortium of 
institutions. Respondants could choose from the following services offered by 
each organization: organization not used or not available; consultation services; 
assessment conferences; training workshops; publications or research reports (1 = 
used; 0 = not used). 

Student Assessment 

Single item. (I A 1) Reflects extent to which institutions collect data on current 
student’s academic intentions or expectations (1 = not collected; 2 = collected for 
some students; 3 = collected for many students; 4 = collected for all students). 

Single item. (I A 2) Reflects extent to which institutions collect data on current 
student’s college-readiness skills (1 = not collected; 2 = collected for some 
students; 3 = collected for many students; 4 = collected for all students). 

Four item factorially-derived scale. (I A 3-6) Reflects the extent to which 
institutions collect data on current students’ cognitive performance: competence 
in major field; general education competencies; higher-order cognitive skills; 
vocational or professional skills (1 = not collected; 2 = collected for some 
students; 3 = collected for many students; 4 = collected for all students). 

Cronbach alpha = .71. 

Three item factorially-derived scale. (I A 7-9) Reflects the extent to which 
institutions collect data on current students’ affective development and 
satisfaction: experiences and involvement with institution; satisfaction with 
institution; personal growth and affective development (1 = not collected; 2 = 
collected for some students; 3 = collected for many students; 4 = collected for all 
students). Cronbach alpha = .68. 

Single item. (I A 10) Reflects extent to which institutions collect data on 
current student’s academic progress (1 = not collected; 2 = collected for some 
students; 3 = collected for many students; 4 = collected for all students). 
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Variable 



Definition 



Post-college assessment 



Civic/social roles 



Timing of data collection 



Number of instruments 



Student-centered methods 



External methods 



Transcript analysis 



External examination 



Interviews of withdrawing 
students 



Three item factorial ly-derived scale. (I A 11,12,14) Reflects the extent to which 
institutions collect data from former students: vocational or professional 
outcomes; further education; satisfaction and experiences with institution after 
leaving ( I = not collected; 2 = collected for some students; 3 = collected for 
many students; 4 = collected for all students). Cronbach alpha = .83. 

Single item. (I A 13) Reflects extent to which institutions collect data on 
former student’s civic or social roles in the community (I = not collected; 2 = 
collected for some students; 3 = collected for many students; 4 = collected for 
all students). 

Nine item additive index. (I A 1-9) Reflects when institutions collect data (I = 
not collected; 2 = collected at one point in time; 3 = collected at entry and while 
enrolled, or while enrolled and at exit; 4 = collected at entry and at exit; 5 = 
collected at entry, while enrolled, and at exit). 

Nine item additive index. (IB 1-9) Reflects student assessment instruments 
(institutionally developed, state provided, and commercially available) used by 
institution to collect ten types of assessment information: student plans or 
expectations; basic college-readiness skills; higher-order cognitive skills; general 
education competencies; competence in major; vocational or professional skills; 
personal growth and affective development; experiences or involvement with 
institution; satisfaction with institution (1= instrument used; 0 = instrument not 
used). 

Four item factorial ly-derived scale. (I C 1-4) Reflects the extent to which 
institutions use innovative or nontraditional assessment methods: performance 
in capstone courses; portfolios or comprehensive projects; observations of student 
performance; individual interviews or focus groups (I = not used; 2 = used in 
some units; 3 = used in most units; 4 = used in all units). Cronbach alpha = 
.61. 

Two item factorially-derived scale. (I C 8-9) Reflects the extent to which 
institutions use assessment methods that data from external constituencies: 
employer interviews or focus groups; alumni interviews or focus groups ( I = not 
used; 2 = used in some units; 3 = used in most units; 4 = used in all units). 
Cronbach alpha = .63. 

Single item. (I C 5) Reflects extent to which institutions use transcript analysis 
to collect student assessment information (1 = not used; 2 = used in some units; 

3 = used in most units; 4 = used in all units). 

Single item. (I C 6) Reflects extent to which institutions use external 
examinations to collect student assessment information (I = not used; 2 = used 
in some units; 3 = used in most units; 4 = used in all units). 

Single item. (I C 7) Reflects extent to which institutions use interviews with 
withdrawing students to collect student assessment information (I = not used; 2 
= used in some units; 3 = used in most units; 4 = used in all units). 
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Variable 


Definition 


Student sub-populations 


Four single items. (I D 1-4) Reflect the use of different assessment methods for 
the following different student populations: a - adult students; b - part-time 
students; c - minority students; d - distance education students (1 - different 
method; 2 - same method). 


Number of studies 


Nine item additive index. (1 E 1-9) Reflects the number of studies institutions 
conduct on the relationship between aspects of students’ institutional experiences 
and performance: course-taking patterns; exposure to different teaching methods; 
patterns of student-faculty interaction; extra-curricular activities; residence 
arrangements; Financial aid and/or employment; admission standards or policies; 
academic advising patterns; classroom, library and/or computing resources (1 - 
conduct study; 0 - do not conduct study). 


Number of reports 


Five item additive index. (I F 1-5) Reflects the levels of aggregation at which 
student assessment data are provided as reports: institution-wide; schools or 
colleges; academic programs or departments; special populations or subgroups of 
students; by course or groups of courses (1 = report provided; 0 - report not 
provided). 



Organizational and Administrative Support Patterns 

Mission emphasis Three item additive index. (II A 1 a-c) Reflects institutions’ mission statement 



Internal purposes 


emphasis on undergraduate education and its assessment: emphasizes excellence 
in .undergraduate education; identifies educational outcomes intended for students; 
refers to student assessment as important activity (1 = yes; 0 = no). 

Four item factorially -derived score. (11 B 3-6) Reflects the importance of internal 
institutional purposes for undertaking student assessment: guiding undergraduate 
academic program improvement; improving achievement of undergraduate 
students; improving faculty instructional performance; guiding resource allocation 
decisions (1 - no importance; 2 = minor importance; 3 = moderate importance; 

4 - very important). Cronbach alpha - .79. 


Accreditation purposes 


Single item. (II B 1) Reflects importance of preparing for institutional 
accreditation self-study as a purpose for undertaking student assessment (1 - no 
importance; 2 = minor importance; 3 = moderate importance; 4 = very 
important). 


State purposes 


Single item. (II B 2) Reflects importance of meeting state reporting 
requirements as a purpose for undertaking student assessment (1 - no 
importance; 2 = minor importance; 3 = moderate importance; 4 = very 
important). 


Administrative and governance 
activities 


Seven item additive index. (II C 1-7) Reflects the number of administrative or 
governance activities used by institutions to promote student assessment: annual 
institution-wide assessment forums or seminars; rewards or incentives for 
administrators promoting use of assessment in unit; incentives for academic units 
to use assessment information; assessment workshops for administrators; board of 
trustees committee addresses assessment; faculty governance committee addresses 
assessment; student representation on assessment committees (1 = yes; 0 - no). 



Variable 


Definition 


Administrative and faculty 
support 


Four item additive index. (11 D 2-5) Reflects the degree to which chief executive 
officer, academic and student affairs administrators, and faculty support student 
assessment (1 = very unsupportive; 2 = somewhat unsupportive; 3 = neutral or 
unknown; 4 = somewhat supportive; 5 - very supportive). 


Type of plan or policy 


Seven single items. (II E 1 a-g) Reflects the institutions plan or policy for 
student assessment: a = formally adopted plan or policy requiring assessment 
activities for aU academic units; b = formally adopted plan or policy requiring 
assessment activities for some academic units: c = formallv adopted plan or 
policy requiring gil academic units to develop their own assessment plan; d = 
formally adopted plan or policy stipulating institution-wide activities to be 
conducted by central committee, office, or officer; e = has no formal plan or 
policy but academic units are encouraged to conduct their own assessment 
activities; f = is currently developing plan or policy; g = does not have an 
assessment plan or policy (1 = yes; 0 = no). 


Formal centralized policy 


Single item. (II E 1 a) Reflects institution has formal institutional plan or 
policy requiring specified student assessment activities of all academic units or 
programs (1 = yes; 0 = no). 


Institution-wide planning 
group 


Single item. (II E 2) Reflects institution has institution-wide group for student 
assessment planning and policy setting (1 = yes; 0 = no). 


Breadth of assessment planning 
group 


Nine item additive index. (II E 3 a-i) Reflects the number of internal members 
included in the institution’s assessment planning group: chief executive officer; 
academic affairs administrator(s)/staff; student affairs administrator(s)/staff; 
institutional research administrator(s)/staff; academic review and evaluation 
administrators) /staff; student assessment administrator(s)/staff; faculty; students; 
other. 


Responsibility for planning 
group 


Seven single items. (II E 4 a-g) Reflect the internal members who have 
executive responsibility for the institution-wide group responsible for planning or 
policy-setting for assessment: a = academic affairs administrator; b = student 
affairs administrator; c = institutional research officer; d = academic review and 
evaluation officer; e = student assessment officer; f = faculty member; g = other (1 
= yes; 0 = no). 


Approval authority 


Eleven single items. (II E 5 a-k) Reflect the internal members who approve any 
changes to institutions assessment plan or policy: a = board of trustees; b = 
chief executive officer; c = chief academic affairs officer; d = chief student affairs 
officer; e = institutional research officer; f = academic review and evaluation 
officer; g = student assessment officer; h = student government; i = academic 
senate or other faculty committees; j = faculty union; k = other (1 = yes; 0 = no). 


Operating responsibility 


Eight single items. (II E 6 a-h) Reflect the internal members who have 
operational responsibility for the institution’s day-to-day assessment activities: a 
= academic affairs administrator; b = student affairs administrator; c = 
institutional research officer; d = academic review and evaluation officer; e = 
student assessment officer; f = faculty member; g = other; h = no one (1 = yes; 0 
= no). 



Variable 

Reporting relationship 



Conducted evaluation 



Evaluations elements 



Definition 

Six single items. (II E 7 a-f) Reflect the individual to whom person with day- 
to-day responsibility reports: a - chief executive officer; b = chief academic affairs 
officer; c = chief student affairs officer; d = institutional research officer; e = 
academic review and evaluation officer; f = other ( 1 = yes; 0 = no). 

Single item. (II F 1 a-d) Reflects if institution has formally or informally 
evaluated its student assessment process (1 = yes; 0 = no). 

Eight single items. (II F 2 a-h) Reflect the elements that were reviewed during 
the institutions assessment evaluation: a = student assessment plan or policies; 
b = structure and responsibility for assessment; c = achievement of intended 
objectives; d = reliability and validity of instruments and methods; e = quality of 
data analysis; f = use of information in institutional decision-making; g = 
problems encountered; h = comparison of costs and benefits (1 = yes; 0 = no). 



Assessment Management Policies and Practices 

Budget decisions Two item additive index. (IV A 3-4) Reflects formal use of assessment 

information in the budget process: to competitively allocate resources among 
academic units; to reward academic units for improvement (1 = yes; 0 = no). 

Computer support Three item additive index. (IV B 2-4) Reflects institutional capacity to collect 

and manage student assessment information: computerized student information 
system includes student performance indicators; student information system 
tracks individual students; student assessment database integrated with other 
institutional databases (1 = yes; 0 = no). 

Access to information Five item additive index. (IV C 1-5) Reflects internal accessibility of 

assessment information on individual students by: institutional research or 
assessment professionals; senior academic administrators; department chairs or 
academic program administrators; student affairs professionals; faculty advisors (1 
= yes; 0 = no). 

Six item additive index. (IV D 1-6) Reflects the number of constituent groups 
to whom student assessment reports are regularly distributed: students; faculty; 
academic administrators; student affairs professionals; employers; general public 
(1 = yes; 0 = no). 

Three item factorially-derived scale. (IV E 1,3,4) Reflects the extent to which 
institutions have policies or practices to promote student involvement in 
assessment activities: inform students about assessment purposes and uses; 
require students to participate in assessment activities; provide students with 
individual feedback on assessment results (1 = not done at all; 2 = done in a few 
departments; 3 = done in some departments; 4 = done in many departments; 5 = 
done in most departments). Cronbach alpha = .69. 



Distribution of reports 



Student involvement 
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Definition 



Variable 

Professional development Four item factorial ly -derived scale. (IV F 2-5) Reflects existence of professional 

development policies or practices on student assessment for faculty and academic 
administrators: provide funds for faculty to attend or present at assessment 
conferences; offer student assessment workshops or consultation for faculty; 
provide assistance (e.g., paid leaves, stipends, course reduction) to improve 
faculty use of student assessment; provide student assessment workshops for 
academic administrators (1 - not done at all; 2 = done in a few departments; 3 = 
done in some departments; 4 = done in many departments; 5 = done in most 
departments). Cronbach alpha = .77. 

Student affairs training Two item factorially-derived scale. (IV F 6-7) Reflects existence of professional 

development policies or practices on student assessment for student affairs 
personnel: require assessment training for student affairs staff; provide student 
assessment workshops for student affairs administrators (1 = not done at all; 2 = 
done in a few departments; 3 = done in some departments; 4 = done in many 
departments; 5 = done in most departments). Cronbach alpha = .84. 

Faculty evaluation Five item factorially-derived scale. (IV G 1-5) Reflects existence of faculty 

evaluation and reward policies and practices related to student assessment: 
promotion evaluation considers evidence of student performance; salary 
evaluation considers evidence of student performance; promotion, tenure or salary 
reviews consider faculty participation in student assessment; promotion, tenure or 
salary reviews consider scholarship on assessment; public recognition or awards 
for faculty use of student assessment (1 = not done at all; 2 = done in a few 
departments; 3 = done in some departments; 4 = done in many departments; 5 = 
done in most departments). Cronbach alpha = .77. 

Academic planning and review Four item factorially-derived scale. (IV H 1-4) Reflects the incorporation of 

student assessment data into academic planning and review processes for: 
academic departments or undergraduate programs; general education or core 
curriculum; courses; student academic support services (1 = not done at all; 2 = 
done in a few departments; 3 = done in some departments; 4 = done in many 
departments; 5 = done in most departments). Cronbach alpha = .84. 

Institutional Uses of Student Assessment 

Educational decisions Ten item factorially-derived scale. (V A 1-5, 8-12) Reflects the influence of 

student assessment information in educational decisions: revision of 
undergraduate academic mission or goals; designing or reorganizing academic 
programs or majors; designing or reorganizing student affairs units; allocating 
resources to academic units; modifying student assessment plans, policies, or 
processes; revising or modifying general education curriculum; creating or 
modifying student out-of-class learning experiences; creating or modifying 
distance learning initiatives; modifying instructional or teaching methods; 
modifying student academic support services (1 = no action or influence known; 

2 = action taken, data not influential; 3 = action taken, data somewhat 
influential; 4 = action taken, data very influential), Cronbach alpha = .83. 
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Definition 



Variable 

Faculty decisions Two item factori ally-derived scale. (V A 6-7) Reflects the influence of student 

assessment information in faculty decisions: deciding faculty promotion and 
tenure; deciding faculty salary increases or rewards (1 = no action or influence 
known; 2 = action taken, data not influential; 3 = action taken, data somewhat 
influential; 4 = action taken, data very influential). Cronbach alpha = .79. 

Institutional Impacts of Student Assessment 

Faculty impacts Four item factorial ly-derived scale. (V B 1-4) Reflects student assessment 

impacts on faculty: affected campus discussions of undergraduate education; 
contributed to faculty satisfaction; contributed to faculty interest in teaching; led 
to changes in teaching methods used (1 = not monitored, do not know; 2 = 
monitored, negative impact; 3 = monitored, no known impact; 4 = monitored, 
positive impact). Cronbach alpha = .79. 

Student impacts Four item factorially-derived scale. (V B 5-8) Reflects student assessment 

impacts on students: contributed to student satisfaction; affected student 
retention or graduation rates; affected student grade performance; affected student 
achievement on external examinations (1 = not monitored, do not know; 2 = 
monitored, negative impact; 3 = monitored, no known impact; 4 = monitored, 
positive impact). Cronbach alpha = .82. 

External impacts Seven item factorially-derived scale. (V B 9-1 5) Reflects student assessment 

impacts on external constituents: affected student applications or acceptance rates; 
affected allocation or share of state funding; affected evaluation from regional 
accrediting agency; affected private fund-raising results; affected success on grant 
applications; affected communications with external constituents; affected 
institutional reputation or image (1 = not monitored, do not know; 2 = 
monitored, negative impact; 3 = monitored, no known impact; 4 = monitored, 
positive impact). Cronbach alpha = .82. 

' Assessment of Teaching and Learning for Improvement and Public Accountability: State Governing. Coordinating 

Board and Regional Accreditation Association Policies andJPractices (Cole et al., 1997) 
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